Point releases for the GNU C Library
The glibc 2.26 release is generally only found in relatively fast-moving distributions at this point. For most users, 2.26 has been without problems, but that is not true for everybody. There have been a few significant regressions in this release that have required fixes; one of those was seen as important enough that the question of creating a 2.26.1 point release was raised. Romain Naour subsequently brought that discussion to the libc-alpha mailing list. Having a point release containing important fixes would be helpful to downstream distributors that want to incorporate those fixes, he said.
Like many projects, glibc fixes bugs first in its repository trunk — the branch that will eventually become 2.27 at the moment. The project does not make point releases, but that does not mean it leaves users of its previous releases entirely out in the cold. A branch in the repository is created for each release and, at the discretion of the manager for that particular release, important fixes are backported and added to that "maintenance branch" post-release. So distributors have a stream of nicely backported fixes for each release; what they don't have are point releases that contain a well-defined set of those fixes.
Some glibc developers strongly question the need for those releases.
Creating a formal release is a significant amount of work, involving
running tests, writing release notes, creating tarballs, etc. A major
release is an indication that a point of stability has been reached and the
code is in a state that is meant to be ready for production use. The
maintenance branches, instead, are meant to always be in such a state. A
point release would just be an arbitrary marker placed in the stream of
maintenance patches, so one might as well just use "git describe"
output to name that point. Thus, Florian Weimer observed that "the difference between
glibc 2.26.5 and glibc 2.26-40
seems rather minor to me
". He suggested that all that was really needed was
a special tag at the beginning of each release branch that would show up in
the output of a git describe command.
Representatives from a few distributions made it clear that they felt differently, though. Arjan van de Ven noted that a release tag helps distributors and users to know what is in a given package:
Andreas Huettel, representing Gentoo, agreed that there is value in having a well-known reference point. Yann Morin, of Buildroot, also agreed, noting that these reference points would be especially helpful to smaller distributors that do not have the bandwidth to follow the maintenance branch closely.
In response to the discussion, Siddhesh Poyarekar, the 2.26 release
manager, said that he would probably do a
2.26.1 release in the near future. He explicitly did not commit to doing any point releases beyond
that one, though. He also said that a point release, if he creates one,
should not be seen as a commitment by the project as a whole to create such
releases
in the future. Carlos O'Donell stressed that last point, saying that "we don't want to set a precedent
that the glibc release manager is always going to do a point
release
".
O'Donell went on to say that the glibc community in general lacks the
resources needed to put together regular point releases. But, he said,
downstream distributors might be able to create those releases with
"a little help
" from the glibc developers. He has promised a
proposal describing how that cooperation might work, but it has not been
posted as of this writing. He has, however, come out in favor of Weimer's special-tag
proposal.
One might be tempted to conclude from this discussion that the need for our release-based software-distribution conventions is waning over time. When every point in a project's history already has a unique name (its Git commit ID), and when growing use of continuous-integration techniques leads to a situation where repository contents can be run at arbitrary points with relatively high confidence, perhaps the release rigmarole can be done away with, at least for point releases. But anybody thinking so should consider the ways in which glibc is unique.
The C library sits between most applications and the kernel. While many users install their own application releases, and a certain number are happy to replace the kernel, it is relatively rare to install a new glibc release. The glibc project moves slowly, new releases tend to have little in the way of exciting new features, and glibc is a rather difficult component to replace so, for the most part, whatever is on the system is good enough for most users. Few people feel the need to be running whatever the latest release is, so they tend to care rather less about glibc point releases than about releases of other components in their systems.
Distributors, instead, pay a lot of attention to the C library. Once they have installed a specific version, they are likely to scrutinize any follow-on patches and will not necessarily accept every change that appears in the glibc maintenance branch. So some distributors, at least, may see little value in point releases for glibc, while others (the smaller ones, perhaps) see some use there.
In other words, glibc may be unique in its relative lack of need for
regular point releases; the rest of the community is unlikely to want to
trade in concise version numbers for random-seeming commit IDs. But, even
in the corner of the community where glibc lives, some users would rather
have the occasional well-defined point release, with the result that glibc
2.26.1 may make an appearance in the near future.
Posted Oct 17, 2017 4:39 UTC (Tue)
by NightMonkey (subscriber, #23051)
[Link] (3 responses)
If this major component of the Linux ecosystem is lacking resources to do regular point releases, can it be inferred that the glibc community as a whole is suffering from lack of interest? Do we have a lurking OpenSSL scenario waiting to bite us in glibc because of human resource starvation? Every multinational corporation on the planet likely has glibc somewhere. Can we pass the hat behind the corporate veils?
Posted Oct 17, 2017 20:28 UTC (Tue)
by nix (subscriber, #2304)
[Link] (2 responses)
It's just that nobody wants to do the relatively boring scutwork of doing point releases when the decision about when to make them is totally arbitrary, since *every commit* on the stable branches is meant to be equally suitable for use.
Posted Oct 18, 2017 8:15 UTC (Wed)
by NAR (subscriber, #1313)
[Link] (1 responses)
Posted Oct 19, 2017 7:30 UTC (Thu)
by nim-nim (subscriber, #34454)
[Link]
As for numbering WillyTarreau posted a good automated recipe. Excellent software engineering as usual, Willy !
Posted Oct 17, 2017 6:32 UTC (Tue)
by pitrou (guest, #118306)
[Link] (8 responses)
I'm a bit surprised by this sentence. Does this mean that tests otherwise don't get run on the maintenance branch?
Posted Oct 17, 2017 8:56 UTC (Tue)
by NAR (subscriber, #1313)
[Link] (5 responses)
Posted Oct 17, 2017 10:20 UTC (Tue)
by smurf (subscriber, #17840)
[Link] (4 responses)
If the maintainers don't want to do that, let some distro people step forward.
Posted Oct 17, 2017 14:51 UTC (Tue)
by NAR (subscriber, #1313)
[Link] (3 responses)
Posted Oct 17, 2017 16:04 UTC (Tue)
by zyga (subscriber, #81533)
[Link] (2 responses)
I think it's very reasonable to question this and if it is indeed an infrastructure problem, solve it. The Linaro community tests countless kernel builds automatically 24/7. On ARM where it still is, arguably, not very easy. Given that a typical Linux distribution has automated builders for all the architectures they support it is not inconceivable to say that we can automatically build and test this on just about everything that matters.
If the problem is elsewhere it should be clearly stated so that interested parties can examine it and come up with a plan.
Posted Oct 17, 2017 16:51 UTC (Tue)
by k3ninho (subscriber, #50375)
[Link] (1 responses)
Tests nobody audits, in this case stuff taking zero human effort, aren't providing information to that question of "should I trust this software, and with what tasks?" They may be out of scope for the contemporary functionality, badly configured so irrelevant, or simply not yielding any or any meaningful output.
It's never zero human effort. An unchecked result isn't worth the time it took to run the tests. It's less effort than doing it all by hand, and can even be set up so that a clean run is trivial to confirm and a failing run includes clear diagnostic information about what broke, how and why, but it's literally never zero work.
We really are that short on community person-power that I'm writing this and not some tests. (Of this, I'm a bit ashamed.)
K3n.
Posted Oct 22, 2017 15:17 UTC (Sun)
by mb (subscriber, #50428)
[Link]
If your automated tests are failing and thus you need humans to look at them, you should better do so.
I don't see how this increases the human work. You will have to handle your failing tests anyway.
Posted Oct 19, 2017 9:35 UTC (Thu)
by swilmet (subscriber, #98424)
[Link] (1 responses)
“Release early, release often.”
“If it hurts, do it more often.”
Posted Oct 26, 2017 9:26 UTC (Thu)
by Wol (subscriber, #4433)
[Link]
Many moons ago, I wrote a Critical Path Analysis system. If I have, say, five people, and I have a bunch of jobs that require a person on site, there is a strict upper limit as to how often I can repeat those jobs.
Oh - and asking those same five people to work longer hours is counter-productive. It's called burn-out - squeezing more hours out usually gives you more billable hours for less work done.
Cheers,
Posted Oct 17, 2017 10:55 UTC (Tue)
by post-factum (subscriber, #53836)
[Link]
Arch has adopted this, for instance, and to my taste, this looks just fine.
Posted Oct 18, 2017 9:56 UTC (Wed)
by nim-nim (subscriber, #34454)
[Link] (1 responses)
Yes, sure, we're all be better of with everyone distributing a different git commit, with non reproducible behaviours/bugs right and left and no possibility of shared QA, shared security audits, shared feedback. It works sooo well for Android to have a soup of commits and branches and not two systems with the same mix. One wonders why Google felt the need to bother with Trebble.
Instead of building sand castles with complex management engines that try to simulate dev behaviour and track what git commit is deployed where, requiring permanent network access to every possible git repo under the sun (since commit ordering can't be deduced from commit ids), massive info leak and non trivial cascading failures when part of the network is down or a bad commit is pushed somewhere, projects would be well inspired to automate the act of releasing a version (starting with adding a git release command).
Commit granularity is good and well for project developers and automated regression testing. It is not sufficient for actual deployment. The time scales, level of project knowledge, update frequencies, and level of risk are too different. Borking a dev build has little consequences, borking something that manipulates actual production data with actual real-world consequences is something else entirely. Just ask Equifax.
A release marks human intent and promise (the project reached a stability point, we don't intend to change it right and left in the near future, we are ready to accept problem reports on this state). There's no way for a purely automated git process to capture and convey this intent. A release is a synchronisation point between different individuals and organisations.
Besides, once you painstakingly deploy one of those git management sand castles, and have perfect visibility on your commit use, it is all perfectly useless. The amount of work needed to identify the security or functional properties of each commit is so huge no one bothers. If something fails it's your fault for drawing a bad ticket, no one is accountable and no one will bother analysing what's wrong in the commit mix you're using. You're usually asked to play commit lottery again with a new commit mix.
So doing a release is teeedious and a “rigmarole” and not needed by the devs themselves. Well, devs develop software. They can build better tools to make releasing less tedious. That's what Linux did when he got fed up with SCM state. Arguing releasing has no function is perfectly disingenuous and non productive.
Posted Oct 18, 2017 15:55 UTC (Wed)
by liw (subscriber, #6379)
[Link]
Posted Oct 19, 2017 5:01 UTC (Thu)
by wtarreau (subscriber, #51152)
[Link] (1 responses)
Posted Oct 20, 2017 11:54 UTC (Fri)
by nim-nim (subscriber, #34454)
[Link]
Anyway the main problem is the lack of "release" notion at Git level, when devs want to use Git exclusively. One can try to workaround it but it will always be fragile and project-specific. I took the time to push the subject upstream.
https://public-inbox.org/git/1290947539.4254.150849603981...
I hope they'll do something.
Point releases for the GNU C Library
Point releases for the GNU C Library
Point releases for the GNU C Library
Point releases for the GNU C Library
Point releases for the GNU C Library
Point releases for the GNU C Library
Point releases for the GNU C Library
Point releases for the GNU C Library
Point releases for the GNU C Library
Point releases for the GNU C Library
"You've got a nice lot of unaudited test output over here. It would be a real shame if the results you received somehow weren't the results you were expecting."
Point releases for the GNU C Library
If your automated tests are passing, no human interaction is needed.
Point releases for the GNU C Library
https://martinfowler.com/bliki/FrequencyReducesDifficulty...
Point releases for the GNU C Library
Wol
Point releases for the GNU C Library
Point releases for the GNU C Library
> distribution conventions is waning over time.
Point releases for the GNU C Library
Point releases for the GNU C Library
Point releases for the GNU C Library