Debian discusses how to handle 2038
At this point, most of the kernel work to avoid the year-2038 apocalypse has been completed. Said apocalypse could occur when time counted in seconds since 1970 overflows a 32-bit signed value (i.e. time_t). Work in the GNU C Library (glibc) and other C libraries is well underway as well. But the "fun" is just beginning for distributions, especially those that support 32-bit architectures, as a recent Debian discussion reveals. One of the questions is: how much effort should be made to support 32-bit architectures as they fade from use and 2038 draws nearer?
Steve McIntyre started the conversation with a post to the debian-devel mailing list. In it, he noted that Arnd Bergmann, who was copied on the email, had been doing a lot of the work on the kernel side of the problem, but that it is mostly a solved problem for the kernel at this point. McIntyre and Bergmann (not to mention Debian as a whole) are now interested in what is needed to update a complete Linux system, such as Debian, to work with a 64-bit time_t.
McIntyre said that glibc has been working on an approach that splits the problem up based on the architecture targeted. Those that already have a 64-bit time_t will simply have a glibc that works with that ABI. Others that are transitioning from a 32-bit time_t to the new ABI will continue to use the 32-bit version by default in glibc. Applications on the latter architectures can request the 64-bit time_t support from glibc, but then they (and any other libraries they use) will only get the 64-bit versions of the ABI.
One thing that glibc will not be doing is bumping its SONAME (major version, essentially); doing so would make it easier to distinguish versions with and without the 64-bit support for 32-bit architectures. The glibc developers do not consider the change to be an ABI break, because applications have to opt into the change. It would be difficult and messy for Debian to change the SONAME for glibc on its own.
Moving forward with the 32-bit time values will, obviously, run into
year-2038 problems, so it makes sense for a system like Debian to request
64-bit
time support—except that there is a lot of code out there that will not
work at all with the newer ABI. Some of it is in binary form, proprietary
applications of various sorts (e.g. games), but there are plenty of
problems for the open-source code as well.
Bergmann scanned the libraries in Debian
and "identified
that about one third of our library packages would need rebuilding
(and tracking) to make a (recursive) transition
", McIntyre said.
He outlined two ways forward that they had come up with.
The first is to rename libraries that need to be fixed to support 64-bit
time values so that there could be two versions of them that could both be
installed on a single system. That entails fixing a bunch of packages and
rebuilding lots of code. "This effort will be *needed* only for the sake of our 32-bit
ports, but would affect *everybody*.
"
The second is to decide which of the 32-bit architectures Debian supports will actually be viable in 2038 and to create new versions of those architectures with different names. There would be two versions of those architecture ports active for, probably, one release and users would not be able to simply upgrade from one to the other. But it would reduce the impact:
He and Bergmann think the second option is the right way to go, but McIntyre was soliciting input from others in the project. Ansgar Burchardt replied that the i386 port, at least, probably should not even take the second path. It should simply continue using 32-bit time values.
If that is the chosen direction for i386, Russ Allbery suggested that the "cross-grading" feature be fully supported. Cross-grading would allow a Debian system to be upgraded to a new architecture, but is currently not for the faint of heart:
But Burchardt is not
convinced there will be enough i386 Debian systems to matter in even
ten years. He showed some numbers from Debian's popularity contest
(popcon) on i386 versus amd64 that suggest i386 will only make up around
0.2% of the total in ten years. "For just avoiding the Y2038 problem,
i386 might sort itself out already
without additional intervention.
" McIntyre thought that
cross-grading support might make an interesting project for an intern, however.
There is work going on in glibc to provide both 32- and 64-bit interfaces
simultaneously, Guillem Jover said,
which could be used to provide a smoother transition without needing a
SONAME change. Bergmann said
that the glibc work is proceeding, but he was not sure that it would
actually help with the transition that much; the problem is that on the
scale of a whole distribution it "adds so much extra work and time before
completion that there won't be many people left to use it by the time
the work is done.;-)
"
Jover pointed to the large-file
support (LFS) transition as something of a model, though he cautioned that
"the LFS 'transition' is not one we
should be very proud of, as it's been dragging on for a very long
time, and it's not even close to be finished
". LFS allows
applications to handle files with sizes larger than a 32-bit quantity
can hold, which parallels the time_t situation. Jover
suggested that fully enabling LFS as part of the time_t transition
might make sense. Bergmann said that LFS support is "a done
deal
", as both glibc and musl libc require using 64-bit file
sizes (off_t) when 64-bit time values are used.
Lennart Sorensen also saw parallels to the LFS transition, but Ben Hutchings thought otherwise:
Similarly, every program that uses wall-clock time will fail as we approach 2038, and the failure mode is likely to be even worse than with LFS as few programs will check for errors from time APIs.
YunQiang Su suggested a third option to consider, which would define 64-bit versions of all of the affected interfaces throughout the distribution, by adding a "64" for the new function and data structure names, then modify packages to use those interfaces over time. A deadline could be set of, say, 2030; after that, anything using the older interfaces would be considered to have a release-critical bug. McIntyre said that it would be technically possible to do so, but there are some major downsides:
2030 is also too late to fix the problem - people are already starting to see issues in some applications. We want to make a difference soon: the earlier that we have fixes available, the fewer broken systems will have to be fixed / removed / replaced later.
Most who commented in the thread seem to see i386 as a dying architecture, at least for Debian on the time scale being considered here (18 years). Florian Weimer summed it up this way:
Bergmann generally agreed with that assessment, but thought it made sense to see how things go for a different 32-bit architecture that probably needs to have a user space that supports 64-bit time_t: armhf. Once that work is done, it may be straightforward to apply it to i386 if that is deemed useful—or a decision could be made to phase out i386 sometime before 2038.
But Marco d'Itri wondered why the picture for armhf was different than that of i386. Bergmann pointed out that armhf is being used for a lot of embedded systems and that is likely to continue. McIntyre also noted that the Civil Infrastructure Platform is based on Debian and is often used with armhf. Bergmann gave a summary of some research he did on the use of the various Arm architecture versions, some of which he expects to still be in use, in 32-bit form, in 2038 and beyond:
The consensus seems to be that the second option, a new architecture name that indicates 64-bit time support, is the right way to go and that armhf makes the most sense as a starting point. In his reply to Jover, Bergmann summed it up this way:
McIntyre is hoping to get started on work soon, so that an armhf port for 64-bit time might perhaps be released with Debian 11 ("bullseye"), which is presumably coming in mid-2021. Bergmann also noted that Adélie Linux has been working on porting its user space for 64-bit times using musl libc; it has a list of open issues that were found:
Even though the 32-bit architectures were largely the focus of the discussion, there is still quite a bit of work that needs to be done for 64-bit systems as well. While the C libraries will soon fully support 64-bit time_t values, they will also require them for the time interfaces; there is plenty of code assuming 32-bit time values buried in user space, but those will presumably be found and fixed over the next few years. We may still see a glitch or three in January 2038 (or before), but it seems like the Linux world, at least, will be pretty well prepared long before the appointed hour.
Posted Feb 19, 2020 21:46 UTC (Wed)
by smoogen (subscriber, #97)
[Link]
(*) Going from the fact that 8 bit and 16 bit processors were really big in embedded even after 'everyone' had moved to 32 bit in the 1990's and 2000's. In fact the chips become more popular when desktops are no longer the main focus because the price per chip goes down greatly. Currently aarch64 is mainly a 'phone' and 'server' chip which makes it a higher price point than a 32 bit arm.
Posted Feb 19, 2020 22:19 UTC (Wed)
by rgmoore (✭ supporter ✭, #75)
[Link] (1 responses)
I think this is a terrible mistake. Even apart from the possibility of machines made today continuing to operate for more than 18 years, computers need to perform calculations related to times in the future. That means problems with 32-bit representations of the time overflowing will start to show up well before the system clock itself overflows. This isn't exactly brain surgery; we know from experience with the Y2K bug that computers will start to encounter errors well before the drop dead date. That knowledge is precisely why we're working on the 2038 bug today rather than waiting until some time closer to 2038; we know systems made today have a reasonable chance of encountering problems from 32-bit times overflowing.
Posted Feb 20, 2020 3:48 UTC (Thu)
by dbnichol (subscriber, #39622)
[Link]
Posted Feb 20, 2020 17:22 UTC (Thu)
by martin.langhoff (guest, #61417)
[Link] (15 responses)
Posted Feb 20, 2020 19:54 UTC (Thu)
by pbonzini (subscriber, #60935)
[Link] (14 responses)
Posted Feb 20, 2020 20:57 UTC (Thu)
by nivedita76 (subscriber, #121790)
[Link] (13 responses)
I used to work for the trading desk of a large bank and we had to convert to 64-bit time years ago. The issue is that certain things do care about time, eg markets have specific opening hours, Fed meeting data and economic data come out at certain times of day, some commodities like electricity or natural gas need intra-day handing etc. Of course this only applies for short-term stuff, but because you write your code to be generic enough to handle both short-term and long-term transactions with the same code, you find one fine day that you can't price a 30-year bond any more because its converting dates into times somewhere deep inside.
Posted Feb 21, 2020 5:20 UTC (Fri)
by ncm (guest, #165)
[Link] (12 responses)
That seems entirely adequate, and finance agrees. Why a seconds count needs to remain signed escapes me.
Posted Feb 21, 2020 6:05 UTC (Fri)
by matthias (subscriber, #94967)
[Link] (8 responses)
Some people like to deal with dates before 1970.
Also, I do not share your pessimism that humanity will menage to extinct itself until 2106. I know it is trying hard, but I still hope that it does not succeed. Thus a change to 64 bit would be necessary anyway, just a few years later, which probably would make the problem bigger, due to the increased amount of code generated until then.
Posted Feb 21, 2020 12:55 UTC (Fri)
by excors (subscriber, #95769)
[Link] (7 responses)
NASA's SPICE system (used for analysing the motion of spacecraft) seems to mainly use Barycentric Dynamical Time (TDB), which (if I understand correctly) is based on a hypothetical clock located at the center of mass of the Solar system (though not in the system's gravity well), with linear scaling to better match the time on Earth (else it would diverge by about 0.5 seconds per year because of relativity). "time on Earth" is basically TAI, which is UTC without the leap seconds. The difference between TDB and TAI wobbles around +/-2 msec as the planets move.
SPICE counts TDB as seconds since the J2000 epoch, which is about a minute before midday UTC on 2000-01-01, where the UTC epoch is ultimately based on astronomical observations from Greenwich in 1958-01-01. Apparently J2000 was introduced around 1984, so it's a rare example of a future epoch.
The implementation uses 64-bit floating point for times, so it should have millisecond precision for the next 300K years. That's not great for astrophysics in general (where you might care about two black holes merging in 0.2 seconds, 1.5 billion years ago), but is presumably good enough for spacecraft.
Posted Feb 23, 2020 6:20 UTC (Sun)
by flussence (guest, #85566)
[Link]
But that's even less likely to happen than embedded systems getting y2038 updates from their manufacturer :)
Posted Feb 23, 2020 21:53 UTC (Sun)
by ras (subscriber, #33059)
[Link] (5 responses)
You are over thinking it. The object of the exercise is not to decide when the big bang happened. It's to ensure no one can want a time before whatever origin they choose. The actual origin they choose won't matter much, because the definition will end up being something along the lines of "876543210000000000 shall represent 1 Jan 2000". Time 0 will be invalid because it represents a time well before the big bang, when time didn't exist.
So you don't have to pick exact time the big bang started - any time before then will do. Turns out the age of the universe in seconds fits into 59 bits, so a signed 64 bit integer representing seconds since some time "before" the big bang is a natural fit. If you want fractions of a second then adding a 64 fractional component seems like the natural extension. A plank time (which could well be the smallest possible unit of time), is 10E-43, so 64 bits is a little generous but the 64/64 symmetry is sooo appealing to my inner geek.
As it happens, this is the almost the time format NTP has been using for the last 30 years. They chose their origin to be a little later than the big bang (1 Jan 1900, in fact), but otherwise it's exactly as I described.
I imagine NTP chose 1 Jan 1900 because our times now will be small numbers. Choosing a big bang origin makes time will be positive numbers with 17 before the floating point which is more cumbersome. But remember the first 8 or so of those 17 digits will always be the same. Even now an experienced Unix sysadmin / programmer can spot a time in a list of values because they live within a certain range. With 128 bit times that will be able to do that with 8 digits of precision, so every time really could start with "87654321".
The other feature a big bang origin is time offsets with invariably be much smaller numbers, so it will be easy to distinguish an offset (or period of time) from an absolute point in time. I've seen bugs arising arising because the programmer got time offsets and absolute times confused, so this is nice.
Posted Feb 24, 2020 15:23 UTC (Mon)
by matthias (subscriber, #94967)
[Link] (3 responses)
Posted Feb 24, 2020 22:05 UTC (Mon)
by ras (subscriber, #33059)
[Link] (2 responses)
LWN needs joke tags for people like me who are slow on the uptake.
> 2^-144 s
Yes, you are right of course. I got my bases confused.
256 bits does indeed have excellent properties. Perhaps the nicest is that if the lifetime of the universe ends up being finite it's likely it will cover the entire period in plank units, which means if could be a true final solution for all of time. But you end up 14 bytes to the left of the implied decimal point and 18 to the right which is downright ugly. I'd prefer the 256 bits on both sides of the decimal point symmetry that 512 bits gives us.
Posted Feb 27, 2020 7:55 UTC (Thu)
by jengelh (guest, #33263)
[Link] (1 responses)
For integer time perhaps, but there are unsexy asymmetric splits in floating point. Which raises the question whether the machine running the multiverse program is actually having a global time counter, and whether that one will be using integer time to begin with.
Maybe the only reason the universe is expanding ever quicker is because it is running with FP somewhere where each step [i.e. `nextafter` C function] means a larger value the closer you approach MAX_FLT. ;-)
Posted Feb 27, 2020 22:57 UTC (Thu)
by ras (subscriber, #33059)
[Link]
Posted Feb 26, 2020 23:42 UTC (Wed)
by JdGordy (subscriber, #70103)
[Link]
Except they messed up the protocol with signed 32bit maths, so when making a request to NTPd you have to be within 34(?) years of *now* which means all sorts of hacks needed to keep working. (designed hardware lifetime for one of the clients was 30+ years without updates).
Posted Feb 21, 2020 8:19 UTC (Fri)
by johill (subscriber, #25196)
[Link]
And even pcap already used 64 bits for the timestamp, just split them into seconds and microseconds/nanoseconds (depends), wasting 2 bits if nanoseconds are used, but the more common is microseconds where 12 bits are wasted. pcapng instead uses a single 64-bit timestamp with an even more variable resolution instead...
Now, this does mean that if you do use nanoseconds, it's only good until shortly before the year ~2555, but ...
Posted Feb 21, 2020 18:18 UTC (Fri)
by nivedita76 (subscriber, #121790)
[Link] (1 responses)
Plus making them unsigned means you need a cast every time you subtract two of them, no?
Posted Mar 2, 2020 15:17 UTC (Mon)
by welinder (guest, #4699)
[Link]
https://news.yale.edu/2015/09/22/living-artifact-dutch-go...
Posted Feb 20, 2020 22:00 UTC (Thu)
by flussence (guest, #85566)
[Link] (5 responses)
Posted Feb 21, 2020 3:44 UTC (Fri)
by nivedita76 (subscriber, #121790)
[Link] (4 responses)
Posted Feb 21, 2020 5:16 UTC (Fri)
by ncm (guest, #165)
[Link] (3 responses)
Posted Feb 21, 2020 6:00 UTC (Fri)
by felix.s (guest, #104710)
[Link]
Until somewhat recently, I used Debian on a machine without SSE2 support. I was quite dismayed that I couldn't run the Rust toolchain on it, even though the compiler target is called ‘i686’.
That the current Debian architecture targetting 32-bit x86 is called ‘i386’ (which currently is i686 in fact) at least has the excuse of maintaining continuity/compatibility. If you go as far with making a clean break to rename the whole architecture, at least don’t pick a misnomer.
Though it’s not like such haphazard removing support for older machines doesn’t make me nervous. I still run Debian on an old 32-bit laptop, though that one at least does support SSE2. I do keep wondering when I’ll be forced to drop it.
Posted Feb 21, 2020 18:23 UTC (Fri)
by nivedita76 (subscriber, #121790)
[Link] (1 responses)
Posted Feb 23, 2020 6:35 UTC (Sun)
by flussence (guest, #85566)
[Link]
Yes that's not what i686 means. But in practice, most other distros seem to take it to imply AMD64 sans Long Mode. If Debian wants to continue to support the subset of x86 CPUs that glibc still supports && have PSE-36 && lack SSE, then I won't stop them. (Things like Rust might, unfortunately, but that isn't an i386-only problem.)
Posted Feb 22, 2020 0:23 UTC (Sat)
by steven676 (subscriber, #41893)
[Link] (2 responses)
> While the year 2000 (y2k) problem is not an issue for us, all Linux
Posted Mar 3, 2020 2:58 UTC (Tue)
by sub2LWN (subscriber, #134200)
[Link]
Posted Mar 11, 2020 13:04 UTC (Wed)
by stevem (subscriber, #1512)
[Link]
Posted Mar 6, 2020 18:31 UTC (Fri)
by AndreyK2013 (guest, #94409)
[Link]
Debian discusses how to handle 2038
Debian discusses how to handle 2038
on the time scale being considered here (18 years).
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
Debian discusses how to handle 2038
> implementations will impacted by the year 2038 (y2.038k) issue. The
> Debian Project is committed to working with the industry on this issue
> and we will have our full plans and strategy posted by the first quarter
> of 2020.
Debian discusses how to handle 2038
Debian discusses how to handle 2038
By the way, for several years we've been fighting against incorrect time_t, having implemented the relevant diagnostic in PVS-Studio: V795. Note that the size of the 'time_t' type is not 64 bits. After the year 2038, the program will work incorrectly.
Debian discusses how to handle 2038