Python time-zone handling

Posted Mar 4, 2020 17:36 UTC (Wed) by kleptog (subscriber, #1183)
Parent article: Python time-zone handling

The fundamental issue with Python datetime handling is that it conflates two separate concepts: (a) you have a time as you see on the clock and you may or may not want to refer to a timezone (b) you want to refer to a particular instant in time (like seconds since epoch), and any timezone attached effects the display and calculations. Python supports the former use-case, but that latter is more often what people want.

This leads to storing of datetime in anything other than UTC being incredibly painful because the way datetimes are stored using (year,month,day,hour,minute,second,tz) is fundamentally ambiguous. The tm_isdst field is just a hack to paper over this. See for example the FixedOffsetTimezone in psycopg2 whose sole purpose it to accurately represent the timestamps coming from the database which actually represent instants in time. Manipulation of datetime with timezones in Python is hard to get right, which is unfortunate for a language which tries to be "batteries included".

We have libraries like Maya, Delorean, Arrow and Pendulum which all try to solve this problem but it would be nice if the standard library at least offered a basic datetime type which acted sanely with timezones.

Python time-zone handling

Posted Mar 4, 2020 19:13 UTC (Wed) by perennialmind (guest, #45817) [Link] (4 responses)

Thank you! Civil time and wall/epoch time are different spaces for which relations exist. They deserve assistance from the type system. Implicitly transforming abstract civil intervals (1 day) to elapsed time (86400 SI seconds) ~~is nuts~~ will inevitably result in subtle errors. Add a civil day to an abstract date? Sure. Add SI seconds to TAI? Sure. Add SI seconds to ISO 8601? With a time zone attached, maybe.

One "pound" sounds the same whether you're talking about force (lbf), mass (lb), or currency (£). With F# style units, you can tell the difference:

The unit of measure '£' does not match the unit of measure 'lb'

It's not obvious how (year,month,day,hour,minute,second,tz) is ambiguous. With the IANA timezone, `tm_isdst` obliquely encodes the UTC offset. The difference operator is ambiguous – because it capriciously selects between wall time and civil time units.

Python time-zone handling

Posted Mar 4, 2020 20:15 UTC (Wed) by mathstuf (subscriber, #69389) [Link] (3 responses)

Suppose I make a timestamp for a future time near a DST transition. What should happen if that DST transition is moved over my timestamp's intended time? You also need to encode either the state of the future DST transitions at the time if you care about wall time or store it as UTC if you want that instant. So basically, you need to also store whether you want that time in the wall clock or instant sense.

Python time-zone handling

Posted Mar 4, 2020 21:27 UTC (Wed) by perennialmind (guest, #45817) [Link] (2 responses)

Yes! Both units and position/coordinate are worth making distinct. If you add SI seconds to a tz-aware civil `datetime`, you're asking for extrapolation with an emphasis on the "civil" over the "time". If I make an appointment for noon six months out in a region that decides on a whim to adopt DST, I'll should still have my noon appointment. That's how it will work out if I'm using iCalendar, since it stores ISO8601 with IANA time zone id. If I want my Starliner thruster to fire at a precise offset, hopefully I'm using something like CLOCK_MONOTONIC, CLOCK_TAI, or Barycentric Dynamic Time.

Python time-zone handling

Posted Mar 4, 2020 22:04 UTC (Wed) by perennialmind (guest, #45817) [Link]

Yeesh. I gave the iCalendar spec too much credit. They got the TZID part right, but didn't include UTC offset or a 'isdst' equivalent, so there are times that can't be represented in that format. Bummer.

Python time-zone handling

Posted Mar 5, 2020 0:39 UTC (Thu) by excors (subscriber, #95769) [Link]

> If I want my Starliner thruster to fire at a precise offset, hopefully I'm using something like CLOCK_MONOTONIC, CLOCK_TAI, or Barycentric Dynamic Time.

CLOCK_MONOTONIC sounds like a bad idea for precise thrusting, since it "is affected by the incremental adjustments performed by adjtime(3) and NTP" (per the man page), so 1 second on that clock may not be 1 second in real time. CLOCK_MONOTONIC_RAW sounds a bit safer.

(Both of those have an unspecified starting point though, so they probably wouldn't have saved Boeing from their Starliner issue where it missed the desired orbit because a clock was off by 11 hours.)