The (initial) glibc year-2038 plan

By Nathan Willis
November 19, 2015

The year-2038 problem will ultimately affect all code that uses 32 bits for time values. Over the past year, the Linux kernel community has begun formally hashing out its plan to transition to 64-bit time support. As a result, other development teams have been observing the kernel discussion and have started to plan their own approaches. The GNU C library (glibc) is one such project, and its developers recently drafted an initial plan of attack.

Albert Aribaud unveiled a proposed plan on the glibc mailing list on October 26, soliciting feedback. Aribaud's proposal sets out a few straightforward requirements for any eventual game plan. As was discussed in 2014, glibc will add a feature test macro that 32-bit applications can set to enable the use of year-2038-safe, 64-bit time functions—although the details surrounding the macro have changed since it was first proposed.

TIME BITS and macros

The interfaces that glibc presents to user code have been the subject of most of the debate thus far. The project had previously considered supporting both 64-bit and 32-bit time functions in the same application.

The first draft of the proposed plan provided both a _TIME64_SOURCE macro and a _TIME_BITS macro. Defining _TIME64_SOURCE would make 64-bit time functions available to the application; then defining _TIME_BITS=64 would cause all time functions to use 64-bit times by default. The _TIME_BITS=64 option is implemented by transparently mapping the standard functions and types to their internal 64-bit variants. Glibc would also set __USE_TIME_BITS64, which user code can test for to determine if the 64-bit variants are available.

A subtle effect caused by using two macros in this manner, though, is that it could enable user code to activate 64-bit time functions, but still default to using the 32-bit versions. In essence, that would allow an application to mix and match calls to (hypothetically) clock_gettime() and clock_gettime64().

On that point, some similarities were cited to the project's addition of the Large File Support (LFS) extension several years ago. The LFS extension provided a _FILE_OFFSET_BITS feature test macro; setting _FILE_OFFSET_BITS=64 would cause glibc to remap off_t and the related functions (e.g., ftello() and fseeko()) to their 64-bit equivalents (off64_t, ftello()64, and fseeko64()). But glibc also provided another option, _LARGEFILE64_SOURCE, that made the 64-bit types and functions available in addition to the 32-bit flavors.

Opinion is sharply divided as to whether or not this was a good idea; code calling the *64 functions is neither portable to other C libraries nor does it conform to standards. The same problem could plague the 64-bit time implementation, with _TIME64_SOURCE and _TIME_BITS=64 being analogous to _LARGEFILE64_SOURCE and _FILE_OFFSET_BITS=64.

Rich Felker recommended not repeating the LFS approach when implementing 64-bit time:

Personally I would rather NOT have this functionality, because my experience from the off_t fiasco was that lots of applications erroneously started using the 64-suffixed functions/types rather than defining _FILE_OFFSET_BITS=64, resulting in messy, non-portable, non-standards-conforming code that took a long time to fix. But I suspect lots of people want it.

Not everyone concurred that the LFS approach was a bad one; Myers noted that the LFS transition, which was trickiest for third-party libraries using glibc, was eventually managed. In a separate email, though, he also expressed doubt that there were many use cases where code would need explicit access to the *64 versions of any functions. Without such use cases, having _TIME_BITS=64 both "enable" and "select" 64-bit time would be simpler.

Aribaud did remove _TIME64_SOURCE from the subsequent drafts of the plan. As things stand now, only the _TIME_BITS macro remains.

Further issues

Some points still up for discussion include whether to mandate that 64-bit time only be supported where 64-bit file offsets are supported (essentially, LFS systems), and whether there should be an attempt to fix 32-bit time functions or simply to implement 64-bit time functions and provide them for those developers wanting a solution.

Not supporting 64-bit time on systems without 64-bit file offsets would primarily serve to reduce the complexity of the implementation—that is, there would only be two permutations to consider (32-bit and 64-bit), rather than four (all 32-bit, all 64-bit, 32-bit time with 64-bit file support, and 64-bit time with 32-bit file support). There are, presumably, only a few non-LFS systems left in the wild, so not supporting them likely has little impact. In light of the project's somewhat contentious history with supporting 32-bit and 64-bit file offsets simultaneously, imposing this restriction is understandable.

The case for not attempting to fix (whatever that might mean) 32-bit time functions is, simply, that developers should know better and fix their code, using _TIME_BITS=64 to enable the new, 64-bit time functions. Any application that persists in using 32-bit time will see incorrect results for times after January 2038, but perhaps that is acceptable. Presumably, such instances would be rare, such as applications for which no source code is available.

As far as interfacing with the kernel is concerned, the basic idea is that glibc will always convert time-related type sizes between kernel space and user space. If the system is running a year-2038-safe kernel, glibc will also convert time values between kernel space and user space. It will not, however, attempt to convert values for a non-year-2038-safe kernel. A bit more nebulous is how to fix platforms that already use 64 bits to store times, but use them to store 32-bit values. Ideally, the general 64-bit time fix should not break these systems. Myers pointed out that the 32-bit time values on these system will need to have an explicit padding field.

Glibc will still need to implement and verify all of the functions and types affected by 64-bit time. Aribaud is still in the process of mapping out the complete set of APIs affected—directly and indirectly—by changing the size of time_t. But once that set of affected pieces has been determined, the question still remains whether 64-bit time should become the default.

Here again, comparisons were made to the LFS transition process. Ultimately, the large-scale transition that changing the default would trigger is a prospect that no one relishes, since it will inevitably impact many libraries and applications—as well as costing developers and users time. But it will have to happen eventually. Felker suggested that announcing a timeline for changing the default from 32-bit to 64-bit time in advance would help avoid many of the LFS transition's headaches.

For now, no such timeline has been discussed. 32-bit time will remain the default for some time to come; making 64-bit time the default will be left for some as-yet-undetermined future date, and the decision will no doubt involve consultation from distributors. There are plenty of additional details that the glibc developers will need to work out in order to fully support 64-bit time, but reaching a rough consensus on the approach to take and on the interfaces to use is a good first step.

The (initial) glibc year-2038 plan

Posted Nov 19, 2015 12:21 UTC (Thu) by RobSeace (subscriber, #4435) [Link] (11 responses)

> 32-bit time will remain the default for some time to come

Only for those saps still on 32-bit architectures... My ancient workstation has been 64-bit for ages, and its time_t has always been 64-bit... I always thought that was going to be the ultimate savior from the 2038 problem: everyone will be running 64-bit machines by then! But, it seems like some people are going to insist on still running in 32-bit mode even then...

The (initial) glibc year-2038 plan

Posted Nov 19, 2015 14:57 UTC (Thu) by corbet (editor, #1) [Link] (5 responses)

Remember that systems being deployed now will still be operating in 2038. Wouldn't it be better if we knew that all those elevator controllers, automotive systems, house monitors, etc. won't break in 22 years?

The (initial) glibc year-2038 plan

Posted Nov 20, 2015 5:19 UTC (Fri) by pr1268 (guest, #24648) [Link] (4 responses)

all those elevator controllers

If ThyssenKrupp and Otis don't upgrade to 64-bits soon, then we won't be able to go higher than Floor 2,147,483,647.

(Sorry, couldn't resist!) ;-)

But, I do have a suggestion: Either go all-in for 64-bit time or not at all (i.e. defined by macro at compile time). If all the FLOSS players (e.g. distros, upstream development, etc.) and 3rd-party developers using glibc all start using the 64-bit time_t (irrespective of whether the arch is 32 or 64 bits), then there'll be plenty of time to sort things out before Y2038. In fact, software vendors could certify their products as being 64-bit time compatible for a compliant system, and explicitly deny support (and claim version non-compliance) for those archs/distros/libraries not participating.

I'm not particularly fond of the idea of mixing-and-matching 32-bit and 64-bit time_t at the ABI level. I've somehow managed to shield myself from the pain experienced by the LFS transition, so perhaps I'm not totally enlightened on what challenges glibc faces with dual-compatibility.

The (initial) glibc year-2038 plan

Posted Nov 20, 2015 16:05 UTC (Fri) by raven667 (subscriber, #5198) [Link] (3 responses)

> If ThyssenKrupp and Otis don't upgrade to 64-bits soon, then we won't be able to go higher than Floor 2,147,483,647.

We'll need to fix this before we build a space elevator. 8-)

The (initial) glibc year-2038 plan

Posted Nov 23, 2015 12:25 UTC (Mon) by robbe (guest, #16131) [Link] (1 responses)

„Space“ is commonly defined to begin at floor 2^15. So ye olde 16-bit microcontroller will work perfectly fine.

Granted, a working space elevetor will probably have to go to GEO, but that's also below floor 2^24.

Even going to the moon only takes 27 bits. But I'd call this a „cable car“ rather than an elevator, and it would have to overcome even bigger problems than the already fantastic space elevator. Speccing a €1 chip rather than a €0.1 chip probably does not even register on the problem scale.

Mars, on the other hand, not only Needs Women, but also 64 bit!

The (initial) glibc year-2038 plan

Posted Nov 23, 2015 14:37 UTC (Mon) by nybble41 (subscriber, #55106) [Link]

> Granted, a working space elevetor will probably have to go to GEO, but that's also below floor 2^24.

A working space elevator would probably need to go about twice that far, so that its center-of-mass is in geosynchronous orbit. Depending on the floor height, that could be slightly over 2^24 floors. (On the other hand, it's doubtful that a real space elevator would have actual floors spaced out every three meters along its length.)

The (initial) glibc year-2038 plan

Posted Nov 29, 2015 6:47 UTC (Sun) by marcH (subscriber, #57642) [Link]

> We'll need to fix this before we build a space elevator. 8-)

The aerospace industry has very high coding standards; not a chance some stupid overflow could ever happen up there.

http://www.around.com/ariane.html

The (initial) glibc year-2038 plan

Posted Nov 19, 2015 16:39 UTC (Thu) by sdalley (subscriber, #18550) [Link] (2 responses)

There are an awful lot of saps still on 32-bit architectures, including almost everyone on linux ARM, android and embedded systems, which of course hugely outnumber linux workstations these days.

My android phone seems to have the epoch set to 01/01/2010. This of course is a reasonable way to kick the can down the road.

Android of course doesn't use glibc anyway.

The (initial) glibc year-2038 plan

Posted Nov 19, 2015 17:14 UTC (Thu) by ScottMinster (subscriber, #67541) [Link] (1 responses)

How do you know that the epoch is from 2010? Running "/system/bin/date +%s" on my Android tablet (in a terminal emulator app) gives a value of 1447952541, which is consistent with an epoch of 1/1/1970. It's possible there is some translation going on in the 'date' program, but how would one know? Write a custom app to call time(2)? Or is this documented somewhere for Android's C library? The Java APIs are defined for 1970, but of course they could be translating from the C calls.

The (initial) glibc year-2038 plan

Posted Nov 20, 2015 0:40 UTC (Fri) by sdalley (subscriber, #18550) [Link]

Actually, I don't know. The factory reset date came up as the first day of 2010, but that's admittedly a stretch to infer that the underlying epoch is actually that and that HTC's (in this case) default customizations didn't select that date, and/or that the calendar clock circuit didn't start at that date.

Unjustified assumption, sorry.

The (initial) glibc year-2038 plan

Posted Nov 21, 2015 5:06 UTC (Sat) by marcH (subscriber, #57642) [Link]

> But, it seems like some people are going to insist on still running in 32-bit mode even then...

Do you expect all Linux/Unix RAM usages to outgrow 4G in the next decade or two?

True: software seems to follow some kind of entropy law where it only ever grows - and is eventually replaced by newer and leaner software when/if needed. Yet 4G seems quite high as the minimum for a usable Unix system in the 2030s...

The (initial) glibc year-2038 plan

Posted Nov 21, 2015 20:24 UTC (Sat) by ykram (guest, #89515) [Link]

You would think so but I think the recent news articles regarding France's Win 3.1 issue is very telling. Fixes exist but adoption seems to be left on the wayside.

Explicity in data types

Posted Nov 19, 2015 13:44 UTC (Thu) by pixelpapst (guest, #55301) [Link] (10 responses)

Whatever the plan, I'd love to (additionally) have a time64_t available - if only to clearly document interfaces that have been future-proofed for 64-bit time values. Think design by contract. (And yes, I'm aware that this causes problems once we need to make time_t 128 bit - but I expect such a step (if ever) to come from the resolution side of things, not because a 64 bit second counter overflows.)

Then again, maybe nobody will attempt to compile C code anymore by 2038, and our preferred systems programming language will provide type conversion operators to handle all possible cases.

Explicity in data types

Posted Nov 23, 2015 13:38 UTC (Mon) by tao (subscriber, #17563) [Link] (9 responses)

"[...] once we need to make time_t 128 bit [...]"

We will never need to.

time64_t can represent 2^63 (time_t is signed) seconds from the epoch.

Our sun is, according to figures from NASA, estimated to be 4.5 billion
years old and to have a further 5 billion years left to live.
So it has a total life span of roughly 10 billion years.

One year is, very roughly, 3 * 10^7 seconds.

Assuming short scale billions (10^9) we get:

10 * 3 * 10^7 * 10^9 seconds, or 3 * 10^(1+7+9) = 3 * 10^17 seconds.

As a comparison 2^63 seconds is roughly 9 * 10^18 seconds.

Now, I dunno about you, but I suspect that:

a.) You and I and humanity as we know it won't exist by the time our sun dies.

b.) If there are any human-descendent beings 2^63 seconds from now, and they use computers,
those computers won't be using operating systems descendent from those we use now...

PS: These calculations assume that the epoch for time64_t
is set to the creation of our sun rather than to 1970-01-01.
Adjusting for the time64_t epoch is easy enough though.

Explicity in data types

Posted Nov 23, 2015 19:02 UTC (Mon) by zlynx (guest, #2285) [Link] (8 responses)

But a truly universal time standard would record time in Planck units instead of seconds or nanoseconds. That is 10e-43 seconds.

So a 128 bit time_t would be useful there.

Explicity in data types

Posted Nov 23, 2015 19:08 UTC (Mon) by nybble41 (subscriber, #55106) [Link] (3 responses)

I think you would have to change more than just the number of bits if you wanted to measure time in anything other than seconds. At that point it wouldn't really be time_t anymore.

Re-defining time_t to avoid artificial limitations

Posted Nov 23, 2015 19:57 UTC (Mon) by liw (subscriber, #6379) [Link] (2 responses)

If we're changing time_t to not be seconds-since-epoch, can we please add support for other universes, where time functions differently from ours?

I am getting tired of doing time and multiverse travel with a vechicle operated by a Posix system.

Re-defining time_t to avoid artificial limitations

Posted Nov 23, 2015 20:30 UTC (Mon) by madscientist (subscriber, #16861) [Link] (1 responses)

You just need to link with libtardis. I think the FSF has a decent free software implementation.

Re-defining time_t to avoid artificial limitations

Posted Nov 23, 2015 21:00 UTC (Mon) by nybble41 (subscriber, #55106) [Link]

> You just need to link with libtardis.

You joke, but such a module actually exists in Haskell:

https://hackage.haskell.org/package/tardis-0.3.0.0/docs/C...

Just try to avoid creating any temporal paradoxes.

Explicity in data types

Posted Nov 23, 2015 20:25 UTC (Mon) by mathstuf (subscriber, #69389) [Link]

Well, if we could ever actually measure time that accurate, sure. You're getting to the point where even light can't get very far in such intervals that even the slightest deviation in wire lengths gets to be an accuracy problem. For example, we've been able to measure on scales of about an attosecond (10¯¹⁸ s) so far and one attolightsecond is about 300 picometers, slightly larger than a single water molecule (280 picometers).

Explicity in data types

Posted Nov 24, 2015 17:12 UTC (Tue) by tao (subscriber, #17563) [Link]

I wouldn't envy the maintainer of tzdata...

Explicity in data types

Posted Nov 29, 2015 18:15 UTC (Sun) by alankila (guest, #47141) [Link] (1 responses)

It is sobering to know that we currently struggle to propagate time in the network at accuracy greater than 10^-3 seconds. It will be a long time before any of the digits after, say, 10^-6 actually represent a physically based measurement.

Explicity in data types

Posted Nov 29, 2015 20:45 UTC (Sun) by flussence (guest, #85566) [Link]

It's easy to get an accurate physical time in microseconds if you're sitting in the same building as the clock source. Maintaining that accuracy while transmitting the value to another latitude is the hard part; I don't think NTP accounts for relativity yet.

The (initial) glibc year-2038 plan

Posted Nov 20, 2015 2:40 UTC (Fri) by gerdesj (subscriber, #5446) [Link] (6 responses)

"Any application that persists in using 32-bit time will see incorrect results for times after January 2038, but perhaps that is acceptable. Presumably, such instances would be rare, such as applications for which no source code is available."

If anyone is using an app now for which the source is not available and intend to still be using it in 2038 then they have between now and then to fix up a plan. That plan can always include changing the rest of the system to keep the app running into the future by changing the kernel and glibc etc or doing strange things like permanently running it at an offset in the "past" and fixing up external systems accordingly.

2038 is about 17 years away and that is way beyond the most LTS of LTS distros. Y2K as a worldwide panic attack did not have that amount of notice (despite being on the cards for just less than 2000 years!)

The (initial) glibc year-2038 plan

Posted Nov 20, 2015 3:01 UTC (Fri) by xanni (subscriber, #361) [Link] (1 responses)

Just less than 1,000 years - there was also a Y1K issue.

The (initial) glibc year-2038 plan

Posted Sep 19, 2017 12:07 UTC (Tue) by temun (guest, #114741) [Link]

> Just less than 1,000 years - there was also a Y1K issue.

There was also Y19H and there will probably be a Y21H.

The term "Y2K" is arguably misleading, suggesting that the problem only happens every 1000 years. But as far as I know the problem was caused by 2-digit years, not 3-digit years.

The (initial) glibc year-2038 plan

Posted Nov 20, 2015 14:48 UTC (Fri) by corbet (editor, #1) [Link] (3 responses)

Well, the whole idea is to avoid the panic phase this time around.

It's worth noting that when I got into this field — the early 1980's — people were well aware of the year-2000 issue. We had plenty of notice, we just didn't do anything about it then. I think it's actually a bit late for us to be fixing this problem, but it's also encouraging that serious work is happening on that front.

Panic phase?

Posted Nov 20, 2015 23:38 UTC (Fri) by pr1268 (guest, #24648) [Link] (1 responses)

Well, the whole idea is to avoid the panic phase this time around.

Was there really a panic about the Y2K problem? Yes, I was there, too, but I don't remember that much concern. Maybe I'm seeing this from the "post-mortem" perspective—either we were (1) panicking over a relative non-issue, or (2) our fears and concerns over this forced us to prepare for it and brace for a possible impact.

But, I see this as another issue: If the fears and panic over Y2K were overblown, then it's possible we could get complacent regarding Y2038.

Those long time-based events (e.g. mortgages, insurance policies, etc.) stored in computers usually don't use Unix time stamps for date storage and calculations (unlike 2-digit years as done in the past)—I personally have seen years as a discrete integer field and occasionally spotted a Julian day (as IEEE double).

Panic phase?

Posted Nov 22, 2015 3:09 UTC (Sun) by giraffedata (guest, #1954) [Link]

Was there really a panic about the Y2K problem?

I remember thousands of person hours spent talking about, writing about, and reading about Y2K. And thousands of person hours spent analyzing systems to see if they had a problem and fixing ones that did. And even fixing ones where the risk didn't really warrant it, just so people wouldn't worry. And it happened more or less at the last minute, with the attendant sense of urgency. I call that a panic.

I guess we'll never know if the failure of the Y2K crisis to appear showed that the panic was unwarranted or that the panic worked.

The (initial) glibc year-2038 plan

Posted Nov 22, 2015 2:30 UTC (Sun) by giraffedata (guest, #1954) [Link]

We had plenty of notice, we just didn't do anything about it then

And because I know many people will read this as indicating we were irresponsible, I'd like to opine that we weren't. We didn't do anything about it because we knew it would be more costly to do something about it than to suffer the consequences of doing nothing. Had we realized there would be a panic, maybe we would have thought differently.

One aspect of the issue that gets overlooked too often is that the cost of making a program Y2K-compliant in 1985 depends upon the likelihood that program will still be in use in 2000. Looking at a 1985 program still in use in 1999, it's easy to overestimate that a priori risk. Think of all the programs that didn't make it that far.

I actually wrote a Y2K-noncompliant program in 1998. Upon careful consideration, I decided the cost of having the program go haywire and have to be restarted on 2000.01.01 multiplied by the probability the program would still be running then exceeded the cost of programming a 4 digit year.

Changing the time default from 32 bits t 64 bits

Posted Nov 22, 2015 20:43 UTC (Sun) by vomlehn (guest, #45588) [Link]

I have a little perspective on migrating from 32 to 64 bits--I was one of the members of the LFS group. I think that went pretty well, though there are still some leftover bits that pop up at unexpected times.

Things are looking good on the migration of time values, but the question of setting a deadline for when 64 bits might be the default can be handled in two phases. Right now, people are discussing simply setting a date, but that has the disadvantage that you don't have time to see what the effects are. Instead, you could decide that in two years (or whatever seems useful) you will survey the state of the technology and set a date for the change of the default. Of course, if things were still murky, you could set another date for seeing how things are going, but it seems unlikely to be needed.