this is sad [LWN.net]

this is sad

Posted Jun 20, 2016 2:02 UTC (Mon) by pizza (subscriber, #46)
In reply to: this is sad by zblaxell
Parent article: Klumpp: A few words about the future of the Limba project

> APT needs minutes of CPU time on a PC-class machine to navigate through a carefully curated (i.e. expensive and error-prone) database of packages labelled according to their dependency relationships. A single-package install on a smartphone-class machine could easily drain a fully charged battery just trying to solve the dependency graph, let alone doing all the IO to perform a non-trivial dependency upgrade.

You both vastly overstate apt's CPU requirements and understate modern smartphone processing and I/O capabilities.

And a random apt package's I/O requirements are most likely better than a random android app's requirements, especially when you consider that modern Android actually *compiles* the app upon installation, and nearly everything interesting going on under the hood (on Android) happens via bog-stock sqlite databases.

to post comments

this is sad

Posted Jun 20, 2016 4:42 UTC (Mon) by zblaxell (subscriber, #26385) [Link] (5 responses)

> You both vastly overstate apt's CPU requirements

You're right, I fudged apt's CPU requirement, but not in the direction you're thinking. I'm also aware of current smartphone processing capabilities even though I didn't state them.

I've seen apt keep an 8-core 3.2GHz machine with 24GB of RAM busy overnight trying to solve a dependency problem. That's with just 22,000 packages on a machine many times more powerful than a current smartphone. Feeding a million apps to apt (assuming we ignore the insanity of downloading a million-app package database) would crush its tiny brain.

sqlite trickle-writes a few pages at a time, usually in response to external stimuli (i.e. it's rate-effectively rate-limited by the user). An apt-get upgrade is gigabytes of I/O all at once, and nothing works right until it's done.

this is sad

Posted Jun 20, 2016 12:28 UTC (Mon) by pizza (subscriber, #46) [Link] (4 responses)

> I've seen apt keep an 8-core 3.2GHz machine with 24GB of RAM busy overnight trying to solve a dependency problem.

That sounds more like there's something wrong with the dependency tree due to broken packages.

Granted, some dep solving algorithms are better than others (witness the significant improvement that dnf brought over yum) but even back in the yum days it still took on the order of a couple of minutes to do the depsolving of a major distro upgrade. (And I might add that apt was noticably faster than yum when doing the same sort of thing..)

Still, this isn't exactly an apples-to-apples comparison; your typical GNU/Linux installation has an order of magnitute more software components installed than a smartphone.

And, I might add, after even a minor point update of Android, the system has to "optimize apps" for the new OS (aka "recompile the bytecode") which takes my tablet about 30 minutes for under 100 "apps", leaving the tablet considerably warmer in the process. There's quite a bit of crap going on there.

this is sad

Posted Jun 20, 2016 17:18 UTC (Mon) by zblaxell (subscriber, #26385) [Link] (3 responses)

> Still, this isn't exactly an apples-to-apples comparison;

If you want to understand why users want apples instead of apt, you have to compare apt to apples.

> your typical GNU/Linux installation has an order of magnitute more software components installed than a smartphone.

The "order of magnitude more software components" is the problem statement. Apt is a symptom of this problem, not a solution. Complexity is not competitive.

If I pick a random desktop machine, it has 128 leaf-dependency packages installed, not even double the number found on a modern smartphone (certainly not an order of magnitude more). Each of these is effectively stand-alone with one or two big dependencies (e.g. ten of them are bits of KDE, one is all of LibreOffice, one is most of xorg, one is Chromium, ssh, rsync, etc). Like a modern smartphone, almost half of these packages turn out to be games.

Apt tracks 2484 separate package objects to support the top 128. If I pick a random package and try to upgrade it, apt informs me it breaks 56 other packages immediately, and those have dependencies too. I've done this dance many times before. There will be well over a thousand packages upgraded before it's done, and I'll have to intervene multiple times during the process as apt will pick a solution from 525600 candidates for upgrading this desktop that isn't the one that Debian-QA tested, so it doesn't work.

Most users (myself included) won't do such an upgrade, because the cost and risk just aren't competitive. It's easier, faster, and more reliable to just debootstrap in a chroot or a new root LV and rebuild the local site configuration manually (that's what tools like etckeeper and puppet are for, after all). This will create a new tree of 2484 moving parts that will, unless there is a radical change of direction in the way Debian manages dependencies, also be too expensive to maintain.

This mess is what failure to say "no" to complexity looks like.

this is sad

Posted Jun 20, 2016 17:48 UTC (Mon) by pizza (subscriber, #46) [Link] (2 responses)

Okay, so you're advocating for the "just replace the system" definition of an "upgrade".

That's all fine and dandy. Except that, in order to set up the system to begin with, someone is still going to have to deal with that complexity. And end-users are going to have to live with whatever choices were made for them.

this is sad

Posted Jun 20, 2016 18:05 UTC (Mon) by rahulsundaram (subscriber, #21946) [Link]

FWIW, there are several projects that already do transactions upgrades. For cloud instances, CoreOS is popular. There is also

https://fedoraproject.org/wiki/Workstation/AtomicWorkstation

Pushing binary deltas of images is easier. Layering packages into a underlying image based update systems is a bit more tricky but it is feasible to do.

this is sad

Posted Jun 20, 2016 19:25 UTC (Mon) by zblaxell (subscriber, #26385) [Link]

Not really advocating, just clarifying the motivation behind the simplified dependency model used in projects like flatpak, Limba, etc instead of apt, libsolv, etc. The optimal model is probably somewhere between those extremes: a mixture of aggregation to simplify the dependency graph for common cases, and isolation to allow individual applications and the host system to resolve dependencies without disrupting each other.

It's a safe bet that most users do want to enable someone else to make an apparently monolithic supportable package (or least an aggregation of components that behave as a single unit) out of thousands of mundane technical choices on their behalf. Those that want to make choices themselves self-select themselves away from the rest of us--they just build what they need from source without bothering anyone.