this is sad [LWN.net]

this is sad

Posted Jun 20, 2016 4:42 UTC (Mon) by zblaxell (subscriber, #26385)
In reply to: this is sad by pizza
Parent article: Klumpp: A few words about the future of the Limba project

> You both vastly overstate apt's CPU requirements

You're right, I fudged apt's CPU requirement, but not in the direction you're thinking. I'm also aware of current smartphone processing capabilities even though I didn't state them.

I've seen apt keep an 8-core 3.2GHz machine with 24GB of RAM busy overnight trying to solve a dependency problem. That's with just 22,000 packages on a machine many times more powerful than a current smartphone. Feeding a million apps to apt (assuming we ignore the insanity of downloading a million-app package database) would crush its tiny brain.

sqlite trickle-writes a few pages at a time, usually in response to external stimuli (i.e. it's rate-effectively rate-limited by the user). An apt-get upgrade is gigabytes of I/O all at once, and nothing works right until it's done.

to post comments

this is sad

Posted Jun 20, 2016 12:28 UTC (Mon) by pizza (subscriber, #46) [Link] (4 responses)

> I've seen apt keep an 8-core 3.2GHz machine with 24GB of RAM busy overnight trying to solve a dependency problem.

That sounds more like there's something wrong with the dependency tree due to broken packages.

Granted, some dep solving algorithms are better than others (witness the significant improvement that dnf brought over yum) but even back in the yum days it still took on the order of a couple of minutes to do the depsolving of a major distro upgrade. (And I might add that apt was noticably faster than yum when doing the same sort of thing..)

Still, this isn't exactly an apples-to-apples comparison; your typical GNU/Linux installation has an order of magnitute more software components installed than a smartphone.

And, I might add, after even a minor point update of Android, the system has to "optimize apps" for the new OS (aka "recompile the bytecode") which takes my tablet about 30 minutes for under 100 "apps", leaving the tablet considerably warmer in the process. There's quite a bit of crap going on there.

this is sad

Posted Jun 20, 2016 17:18 UTC (Mon) by zblaxell (subscriber, #26385) [Link] (3 responses)

> Still, this isn't exactly an apples-to-apples comparison;

If you want to understand why users want apples instead of apt, you have to compare apt to apples.

> your typical GNU/Linux installation has an order of magnitute more software components installed than a smartphone.

The "order of magnitude more software components" is the problem statement. Apt is a symptom of this problem, not a solution. Complexity is not competitive.

If I pick a random desktop machine, it has 128 leaf-dependency packages installed, not even double the number found on a modern smartphone (certainly not an order of magnitude more). Each of these is effectively stand-alone with one or two big dependencies (e.g. ten of them are bits of KDE, one is all of LibreOffice, one is most of xorg, one is Chromium, ssh, rsync, etc). Like a modern smartphone, almost half of these packages turn out to be games.

Apt tracks 2484 separate package objects to support the top 128. If I pick a random package and try to upgrade it, apt informs me it breaks 56 other packages immediately, and those have dependencies too. I've done this dance many times before. There will be well over a thousand packages upgraded before it's done, and I'll have to intervene multiple times during the process as apt will pick a solution from 525600 candidates for upgrading this desktop that isn't the one that Debian-QA tested, so it doesn't work.

Most users (myself included) won't do such an upgrade, because the cost and risk just aren't competitive. It's easier, faster, and more reliable to just debootstrap in a chroot or a new root LV and rebuild the local site configuration manually (that's what tools like etckeeper and puppet are for, after all). This will create a new tree of 2484 moving parts that will, unless there is a radical change of direction in the way Debian manages dependencies, also be too expensive to maintain.

This mess is what failure to say "no" to complexity looks like.

this is sad

Posted Jun 20, 2016 17:48 UTC (Mon) by pizza (subscriber, #46) [Link] (2 responses)

Okay, so you're advocating for the "just replace the system" definition of an "upgrade".

That's all fine and dandy. Except that, in order to set up the system to begin with, someone is still going to have to deal with that complexity. And end-users are going to have to live with whatever choices were made for them.

this is sad

Posted Jun 20, 2016 18:05 UTC (Mon) by rahulsundaram (subscriber, #21946) [Link]

FWIW, there are several projects that already do transactions upgrades. For cloud instances, CoreOS is popular. There is also

https://fedoraproject.org/wiki/Workstation/AtomicWorkstation

Pushing binary deltas of images is easier. Layering packages into a underlying image based update systems is a bit more tricky but it is feasible to do.

this is sad

Posted Jun 20, 2016 19:25 UTC (Mon) by zblaxell (subscriber, #26385) [Link]

Not really advocating, just clarifying the motivation behind the simplified dependency model used in projects like flatpak, Limba, etc instead of apt, libsolv, etc. The optimal model is probably somewhere between those extremes: a mixture of aggregation to simplify the dependency graph for common cases, and isolation to allow individual applications and the host system to resolve dependencies without disrupting each other.

It's a safe bet that most users do want to enable someone else to make an apparently monolithic supportable package (or least an aggregation of components that behave as a single unit) out of thousands of mundane technical choices on their behalf. Those that want to make choices themselves self-select themselves away from the rest of us--they just build what they need from source without bothering anyone.