DNF5 delayed

Posted Aug 18, 2023 16:08 UTC (Fri) by rahulsundaram (subscriber, #21946)
In reply to: DNF5 delayed by re:fi.64
Parent article: DNF5 delayed

> The main actual rewrite *from scratch* is the currently-Python frontend, which is where the missing plugins and such mentioned in the post come in

Yep. This looks like substantial refactoring but not a from scratch rewrite.

https://dnf5.readthedocs.io/en/latest/changes.html

Dnf4 already used libdnf and libsolv. That part hasn't changed. Dropping Python entirely on the frontend is new and is significantly faster and uses less memory (in a big part because of better metadata processing - not having to download and load the full file index of everything into memory) but the configuration format, options mostly seems to be retained from the Yum days. I suspect the disruption is mostly for those integrating with the new API (Ansible, Mock, plugins etc) and not for end users.

DNF5 delayed

Posted Aug 20, 2023 11:51 UTC (Sun) by gmgod (guest, #143864) [Link] (4 responses)

> Dropping Python entirely on the frontend is new and is significantly faster and uses less memory

That is very misleading. The python wrapper is very thin. I have personally profiled dnf to see if there were low hanging fruits on the python side to make things faster. The answer is no: dnf spent more than 95% of its time in C parts (libresolve and libdnf). I can't remember the actual number but it was in the very high 90s, even when metadata were already downloaded.

What makes dnf5 better are better algorithms (as implemented in libdnf5).

And the main reason they got rid of python is so that it's not pulled as a dependency in small footprint systems. Dnf5 still has a problem with memory management on those same low-footprint systems though... I did not measure it precisely but even just with "top" you can see that whatever dnf is using, it's dwarfed by metadata loading.

They tried to go too fast by reimplementing the frontend from scratch to get rid of python asap and improve the backend at the very same time but both are huge projects... It would not have been a bad idea to incrementally improve dnf4...

DNF5 delayed

Posted Aug 20, 2023 15:36 UTC (Sun) by rahulsundaram (subscriber, #21946) [Link] (3 responses)

> That is very misleading. The python wrapper is very thin

Yes, it is very misleading when you cut off the part where I attributed to the speed and memory gains to the changes in metadata processing and not the language itself.

DNF5 delayed

Posted Aug 21, 2023 23:16 UTC (Mon) by gmgod (guest, #143864) [Link] (2 responses)

Don't double down on bs. The subject of your sentence was "dropping python". At best your assumptions were wrong, at worse you just did not express yourself clearly and meant to say dropping python is new and the new metadata processing is both faster and use less memory.

DNF5 delayed

Posted Aug 21, 2023 23:30 UTC (Mon) by rahulsundaram (subscriber, #21946) [Link] (1 responses)

> Don't double down on bs. The subject of your sentence was "dropping python"

There was no such subject but I understand your confusion since you read it that way.

DNF5 delayed

Posted Aug 22, 2023 0:17 UTC (Tue) by rahulsundaram (subscriber, #21946) [Link]

> There was no such subject but I understand your confusion since you read it that way.

On second thought, it wasn't the subject that confused you but omitting things written as part of the same sentence - "in a big part because of better metadata processing - not having to download and load the full file index of everything into memory".

There was no implied causation between the switch of the language and that work. In fact, Yum actually did handle this part better than Dnf 4.