The plan for merging CoreOS into Red Hat

Posted May 12, 2018 1:01 UTC (Sat) by lsl (subscriber, #86508)
In reply to: The plan for merging CoreOS into Red Hat by nim-nim
Parent article: The plan for merging CoreOS into Red Hat

> And Go developers are so addicted to not propagating fixes the next-gen Go system to manage version state¹ will “innovate” by automatically choosing the most ancient dependency version possible unless forced otherwise. Integrating fixes is work, better not do it and no one will be the wiser. Forking is a built-in Go property. Sharing fixes and changes, not so much.

It's a bit more nuanced than that. For direct module imports, vgo will actually choose the most-recent version available. It's only for transitive dependencies that the Minmal Version Selection algorithm developed for vgo may choose a version such that you can call it "most ancient".

The actual innovation in vgo is somewhere else, though: not that long ago, no one thought you can get away with a package manager without tackling the NP-complete problem that lurks within version selection and causes package managers to employ weird heuristics, full-blown SAT solvers or (most likely) both.

The MVS algorithm implemented by vgo does (naturally) impose some constraints on programmers but they're workable ones. Its simplicity is very refreshing. Simple enough to perform it in your head. Iterate over the list of declared dependencies, for each module keeping track of the newest version that was actually referenced.

This predictability is a really nice property. There's also an obvious way to get a newer version of some module: require it. Contrary to most other systems, this is guaranteed to not result in unsatisfiable version requirements.

The plan for merging CoreOS into Red Hat

Posted May 12, 2018 4:19 UTC (Sat) by nevyn (guest, #33129) [Link] (4 responses)

> no one thought you can get away with a package manager without tackling the NP-complete problem that lurks within version selection and causes package managers to employ weird heuristics

Eh, this is a weird Apples/Oranges thing IMO. I understand why pip/gems decide to call themselves a package manager, even though they do very different jobs than normal package managers ... but go just isn't one. Very few people are building N different go projects at the same time, and any that are should be using containers anyway at this point in time. This is hardly the same task as having to install httpd and sshd on the same machine (even when you take out all of the main parts of package managers that pip etc. don't even bother with).

The plan for merging CoreOS into Red Hat

Posted May 12, 2018 12:55 UTC (Sat) by lsl (subscriber, #86508) [Link]

Yeah, all this stuff (whether vgo, pip or Rust's cargo) is very different from system package management tools like zypper/dnf/rpm or apt/dpkg.

The vgo solution only works in the context of language-specific dependency installation tools, where you don't have to deal with arbitrary software written by people that don't give a shit about the conventions you'd like to establish.
Things are easier on the language-specific side: Python programmers generally care whether their code works well with pip just as most Go programmers want their code to play well with vgo. They are going to adjust their versioning and release practices to make that happen.

The plan for merging CoreOS into Red Hat

Posted May 15, 2018 9:21 UTC (Tue) by nim-nim (subscriber, #34454) [Link] (2 responses)

Containers just compound the terrible inertia in propagating fixes in Go. What you think are single purpose containers that ship a single app are actually single purpose containers that ship a single app composed of aa private copy of hundreds of other Go projects in its GOPATH.

The container + private copy thing means you can avoid dealing with code changes in the code of those other projects.

But avoiding dealing with code changes also means avoiding propagating fixes.

The plan for merging CoreOS into Red Hat

Posted May 15, 2018 12:31 UTC (Tue) by liw (subscriber, #6379) [Link] (1 responses)

It's 2018, where is the +1000 button on LWN? This is why we don't have flying cars, personal jet packs, and fluffy little robot kittens who fulfil our every need: lack of upvote buttons on LWN. Sheesh.

In other words, I agree with nim-nim on embedding code copies on containers. We have a problem and we need to solve it.

The same problem occurs in other contexts as well, when embedding dependencies are used. There's a clear need for a solution to this problem that makes sure security fixes, and other fixes to sufficiently grave bugs, get smoothly and swiftly and securely distributed to all the embedded copies.

In a way it's similar to what distributions like Debian do for their stable releases: packaged software is not updated to every new upstream version, fixes are backported to the versions in the stable release. I fear the way Debian does this is highly labour intensive, and probably doesn't scale.

Note that technical solutions are not enough. There is also a need for the software developers, sysadmins, and users to understand the issues, and to use whatever solutions there are.

I don't have the solution, but I do see the problem.

The plan for merging CoreOS into Red Hat

Posted May 15, 2018 23:29 UTC (Tue) by pabs (subscriber, #43278) [Link]

> I fear the way Debian does this is highly labour intensive

The Debian security team has automated some aspects of their patch backporting:

https://debconf17.debconf.org/talks/166/