LWN: Comments on "Linux's missing CRL infrastructure"

Change address on every DHCP renewal

farnz — Wed, 03 Sep 2025 09:05:36 +0000

YouTube is not a good test for seamless switching at the network layer, since it makes many small requests, and copes well with connection drop (by design - YouTube is meant to work even on bad networks).

YouTube (and other streaming media platforms) work by stitching together a lot of small (typically 10 seconds or so) clips at the client side, and YouTube is set up so that if you download the first part of a clip, then change network, it's possible to download the rest of that clip with a range request, instead of a full redownload.

It also adapts to network conditions by using lower bitrate clips if requests are either not completing in full, or taking too long (since 10 seconds at 400 kbit/s is less data to transfer than 10 seconds at 2 Mbit/s).

To test properly, you'd need a server side under your control, and you'd be watching for connection drops of something long-lived like an SSH connection. And, of course, the best fix is to not depend on long-lived connections to begin with - you're in the best possible place if connections can drop freely without issue, and you always find systems by mDNS or similar, not IP.

Change address on every DHCP renewal

vonbrand — Tue, 02 Sep 2025 14:13:15 +0000

My phone switches seamlessly between WiFi and 5G, even in the middle of e.g. watching YouTube. Haven't tried changing WiFi connection mid-stream, however.

Interesting suggestion

SLi — Sun, 31 Aug 2025 22:04:01 +0000

Reading the issues in the various projects (all pretty much closed as not in scope), I found this comment quite interesting:

https://github.com/openssl/openssl/issues/28186#issuecomm...

Basically, it suggests that the only reasonable way ahead could be to build some cooperation in how distros do this: Currently there are apparently three ca-certificate distro packages (Fedora, OpenSuse, Debian), all somehow derived from the Mozilla certificate packs. And Mozilla "apparently" doesn't want to be the upstream, and there's a cryptic (but I trust warranted) "Definitely do not pick Debian's".

All this sounds to me like there's some mapping to be done in the social space. Things that are quite nonobvious to me:

1. In what way are the Mozilla specific attributes critical for security (as mentioned in the article, that stripping them is insecure)?
2. What do the three distros do differently, and why? I'd hope the maintainers even talk informally every now and then, although apparently a mid-stream "adapted from Mozilla" project does not exist.
3. What does it mean that Mozilla, allegedly, doesn't want to be the upstream? Probably that their focus is Firefox and other Mozilla products. Hopefully not that they'd go to much lengths to sabotage anything. Could beneficial-to-everyone solutions exist?

And, of course, that still leaves the question of whether this realistically solves things that are not solved by shorter certificate lifetimes. I'd certainly hope so, because 47 days is a long time, but the reality seems to be messy.

(There was also a suggestion of p11-kit as a possible upstream in an earlier comment in that issue. I have little idea what that even is.)

GnuPG's dirmngr can to CRLs

ber — Fri, 29 Aug 2025 16:06:42 +0000

Note that the dirmngr coming with GnuPG (since many years), can do CRLs and is a daemon serving requests, it maybe a basis for a cache.

See https://www.gnupg.org/documentation/manuals/gnupg/Invokin...

Short lived certificate

raven667 — Fri, 29 Aug 2025 14:37:23 +0000

Do hosts not *also* get a stable management address, either EUI-64 based on the MAC (ff:fe old style) or the newer RFC7217 "stable privacy" address based on network prefix, interface, SSID, machine UUID? The temporary privacy addresses should be used for outbound connections based on their preferred lifetime but if you run services the management address should be usable, `scope global dynamic mngtmpaddr noprefixroute` vs `scope global temporary dynamic`.

Change address on every DHCP renewal

farnz — Fri, 29 Aug 2025 10:06:47 +0000

Sure, and we know that the best way to ensure that humans actually learn that lesson is to not let things appear to work for a long period before breaking due to a human error. Humans need feedback fairly close in time to the mistake in order to learn from their errors. Humans are also guaranteed to have a non-zero undetected error rate - from 0.009% for trivial tasks, to 30% for very difficult tasks - and we need feedback to get us to detect the errors.

That's been a very hard lesson for the commercial aviation and nuclear industries to learn - but we don't have to relearn it from scratch, we can learn from their mistakes.

Change address on every DHCP renewal

paulj — Fri, 29 Aug 2025 09:28:30 +0000

At a certain point, you just have to teach humans how to work with the system as it is - while waiting for the AGI that will anticipate your every need, before you even know them.

fetch-crl

amadio — Fri, 29 Aug 2025 07:45:55 +0000

Some communities (e.g. EGI, WLCG), use a tool called fetch-crl to manage CRLs. It's available both in EPEL and in Debian.

Short lived certificate

Cyberax — Thu, 28 Aug 2025 19:13:05 +0000

That access point already has the client's MAC address and can NAT the privacy IPv6 addresses to a non-privacy IPv6.

Change address on every DHCP renewal

farnz — Thu, 28 Aug 2025 16:50:50 +0000

I'm one of the people who's pointed out that IPv6 has standardised support for rotating addresses :-)

And yes, there's no good way to do this in IPv4; which means that you've got a choice between doing it badly (thus ensuring that foul ups break near to the time of the foul up) or not doing it at all (and dealing with the fallout when there's a cascade failure that runs into the foul up).

But there's a general tendency for humans to assume that everything that works is working as intended, and that no foul ups have happened - after all, if a foul up had happened, things would stop working. Any time someone says "I've never had a problem doing this before", you're hearing "this used to work, therefore it must be the right thing to do, and not something that happened to work coincidentally".

This leads to a soft-skills requirement on technology - things that work should either be the right thing, or should break quickly so that the person setting it up doesn't go "this has been working for months - you must have done something". Otherwise, the technology gets the blame for the human problems.

Short lived certificate

NYKevin — Thu, 28 Aug 2025 16:46:53 +0000

That would be a vulnerability - an open WiFi hotspot, that many devices will automatically connect to by default, should not be allowed to demand a trackable IP address (at least, not without the user's consent).

Change address on every DHCP renewal

paulj — Thu, 28 Aug 2025 15:29:17 +0000

In IPv4 there isn't a good, generally agreed way to rotate addresses - without breaking ongoing connections.

As others have pointed out, in IPv6 there is standardised support, and it would be possible. Don't know if there are common DHCPv6 servers that do it - should be possible though. There are ISP systems that deliberately change IP addresses at regular intervals, to stop residential connections from benefiting from static addresses.

Change address on every DHCP renewal

Wol — Thu, 28 Aug 2025 14:08:36 +0000

> That second point means that you need the technical process to back the human process - if the human fouls up (and they will, at some point), you need the technical process to make sure that the foul-up becomes clear quickly.

And what you DON'T do (cit Dick Feynmann, Challenger disaster) is leave a couple of jobs in the system for the human to do, to make them feel involved. Either automate them out of the process entirely, or involve them as much as you can.

I think most recent airline disasters are down to the fact that even very experienced pilots with thousands of hours in their log books, have actually spent very little time *at the controls*. On a ten-hour flight, George probably accrues 9.5 hours, with the humans acquiring half an hour real experience between them. So when something goes wrong, they actually have very little knowledge/feel of *how to fly*. Which in an emergency is an absolute necessity!

Cheers,
Wol

Change address on every DHCP renewal

farnz — Thu, 28 Aug 2025 13:50:40 +0000

There is a way to introduce feedback, however; if the host changes address frequently, then instead of the pattern being "quick temporary fix done, new fire comes up so it's never redone properly, system fails 2 years later, blame the system", it's "quick temporary fix done, new fire comes up so it's never redone properly, system fails tomorrow, remember the quick temporary fix and make it permanent".

The technical problem is that you're not getting feedback that you've fouled up in a reasonable timescale after making the mistake; the system cannot know that you've fouled up, but it can be reworked so that either it's impossible to foul up (address space large enough that all addresses are deterministic, thus static), or so that the system breaks reasonably soon after you fouled up, not years later (addresses either static, or forcibly changed every 24 hours).

Change address on every DHCP renewal

paulj — Thu, 28 Aug 2025 13:40:07 +0000

There is no way for the system to know the human fouled up. How is a host or a DHCP server, or some combination of these and/or other distributed system supposed to - in a generalised way - know that a human has recorded the dynamically assigned IP address of a host somewhere, with the expectation that it will be stable, under the constraint that the address is signed from a small, finite pool (i.e., the IPv4 case)?

I can't think of any way to solve that problem.

The only thing I can see is that you attack that limiting constraint, which can - for many purposes (and almost definitely if we're talking "a human could write it down somewhere", given how that rate limits things) - be achieved by going to an IPv6 network and using some deterministic stateless address assignment.

Change address on every DHCP renewal

farnz — Thu, 28 Aug 2025 13:23:58 +0000

The technical problem is "human did not get actionable feedback when they failed to follow the process".

There's two things coming into play here:

Humans will take shortcuts without considering the system as a whole, and unless you have a feedback mechanism to catch this, it will be blamed on the system, not the humans who made a mistake - not least because the person who did the "quick, temporary" fix might well have left the company, so there's no way to retrain them, but the system is still here, and broken.
No matter how good your training and processes are, there will be a non-zero human error rate. Per this table from a journal article in the nuclear industry, you can expect a human error probability on a task between 0.009% for trivial tasks, going up as far as 30% on hard tasks.

That second point means that you need the technical process to back the human process - if the human fouls up (and they will, at some point), you need the technical process to make sure that the foul-up becomes clear quickly.

It's similar to kernel development in that regard; if I tell Linus "hey, latest rc doesn't work, rc1 did, this is the symptoms", I'm quite likely to get a diagnosis of the problem quickly, and a fix. If I say "hey, 6.7 doesn't work, latest rc doesn't work, 3.2 did, this is the symptoms", I'm not going to get a huge amount of sympathy.

Local OCSP?

daroc — Thu, 28 Aug 2025 12:34:28 +0000

As I understand it, macOS centralizes this check through their Keychain service. An app makes a request to know whether a certificate is valid, and the service walks the chain of trust to a trust anchor, checking revocation at each step along the way.

And OpenSSL already supports this kind of thing — you "just" have to enable CRL checking and tell it where to find the CRL file. So yes, you're totally right that this is more of a social problem than a technical one. In my experience, the open source community is generally quite good at solving technical problems, so all the problems that remain so for long are social.

Change address on every DHCP renewal

paulj — Thu, 28 Aug 2025 12:19:39 +0000

I don't see a technical problem, I see a human problem: "How do we get humans to formally record an expectation with regard to the assignation of a limited resource, so the wider system can grant it?". It's a training and process problem.

You either need (near) infinite addresses, or you need to train your users to follow a process.

Forgot distros?

zdzichu — Thu, 28 Aug 2025 08:19:30 +0000

On Fedora, Java stack uses system CA list: https://docs.fedoraproject.org/en-US/quick-docs/using-sha...

Forgot distros?

taladar — Thu, 28 Aug 2025 07:21:17 +0000

Usually there is (at least) two because the JVM insists on doing its own thing.

Forgot distros?

zdzichu — Thu, 28 Aug 2025 06:16:33 +0000

> Most [applications] fill the content of these root stores by copying Mozilla's root store

Uhm, no? Most applications do not deal with cert stores, because there is one store provided and updated by the distribution. It often comes from Mozilla (https://fedoraproject.org/wiki/CA-Certificates , https://tracker.debian.org/pkg/ca-certificates ), but distros are competent enough not to botch it.

I don't see why CRL couldn't be managed by distros the same way.

Local OCSP?

pj — Wed, 27 Aug 2025 21:13:47 +0000

It's mentioned that other OSs handle this... what do they do? I can think of multiple possibilities off the top of my head:

1. maybe decide/define a well-known location for CRLs, like we do for certs and cert chains. Or maybe add a layer of indirection by defining where to find a pointer to said location (config file, soft link, whatever) If FHS changes are in the works, maybe these things could show up there?
2. maybe promulgate a local OCSP server that can be the cache. Then all the apps that talk OCSP can just talk to localhost, kind of like how talking to multiple DNS upstreams is resolved by running a local nameserver.
3. maybe CRL support could get rolled into OpenSSL such that cert verification (which already is usually its own flag, I believe?) now does CRL things as well?

This seems more a social problem than a technical one.

Mozilla's CRLite

jag — Wed, 27 Aug 2025 19:33:14 +0000

Any solution on Linux should probably use CRLite:

https://hacks.mozilla.org/2025/08/crlite-fast-private-and...

Change address on every DHCP renewal

farnz — Wed, 27 Aug 2025 17:35:50 +0000

Absolutely - the root cause of the problem is humans assuming that because they've never seen something change, it will never change. The technical question is "how do you make sure that, without unacceptable disruption, IP addresses change unless configured to stay static?", so that humans see them change and know that this is normal.

And IPv6 makes the solution space much easier to work in; you're expected to have multiple addresses per host (indeed, DHCPv6 explicitly supports this when it's assigning addresses, not SLAAC or manual config), and you've got much more address space to work in, so that you can have self-organising stable addresses as you describe.

Change address on every DHCP renewal

paulj — Wed, 27 Aug 2025 17:31:13 +0000

Hmm, your problem lies in the domain of humans, more than the machine. The machine has a way for you to express what you want. ;)

IPv6 may help fix this issue. It has a local, on-link address space large enough to let clients use their own address suffix with (next to) no fear of conflict (and having to pick something else). There are at least 2 ways an IPv6 host can give itself a stable address, without having to rely on the server a) use a MAC address derived suffix; b) using a "stable private" address.

Change address on every DHCP renewal

farnz — Wed, 27 Aug 2025 16:20:50 +0000

While you've got the problem correct, you're missing one key point: the clients appear to have static IPs from the perspective of the users of the system; the IPs are technically dynamic, but can stay unchanged for years.

The correct fix is to put a static MAC→IP mapping in config for anything that's hard coded. However, that leads to a human problem; users hard-code the IP as a "temporary quick fix", and thus never ask the DHCP server admin to put a static MAC→IP mapping in place (because this is temporary and will be removed soon), and because the "dynamic" address never changes, this "temporary quick fix" can become semi-permanent because nobody gets round to doing the "proper" fix, or asking for a static address to be configured in DHCP.

So, how do you ensure that the "temporary quick fix" cannot work for more than a day or so, ensuring that the person who did a "temporary quick fix" still remembers what they did when it next breaks?

Change address on every DHCP renewal

paulj — Wed, 27 Aug 2025 15:54:39 +0000

The scenario of the DHCP server starting up with config, but having lost lease state is the norm in OpenWRT. It's generally running on small APs and consumer routers out of RO flash with a tmpfs for most runtime state. If you reboot an OpenWRT router, clients will either DHCPDISCOVER or DHCPREQUEST, and the OpenWRT router will give them their IP. (I assume dnsmasq does some kind of DAD - seems like it does a ping-check for at least DHCPDISCOVER before OFFER, reading the internet).

If a host has been assigned a static IP in the config, that's what it gets.

Your problem seems to be that a host had an IP, from a dynamic pool, and a human and/or software on that host then took a dynamically assigned IP and hard-coded it in places. Relying on the DHCP server to maintain persistent state of the dynamic IP assignment, and assigning the same (dynamic) IP to hosts. ?

Firstly, the DHCP server /could/ give out the address the client requests, if not in use. Like dnsmasq does on OpenWRT. If the client still has the address.

If the client has removed its IP, cause of whatever timeout, and doesn't know to request it cause that state is gone, and the server doesn't have that state either, well... if this kind of a thing is a critical issue, it seems like the correct fix is to just configure a static IP in the DHCP server config, if your hosts are relying on static IPs?

Change address on every DHCP renewal

Wol — Wed, 27 Aug 2025 15:34:31 +0000

Given that the hosts file has to be updated and shipped around, regenerate the DHCPd persistent state by importing the hosts file and say "if the requester says 'my name is X', look up X in the hosts file and return that IP if found".

Dunno how it worked :-) but one system I worked on, you put the hostname in /etc/hostname, and added the host/IP to /etc/hosts, and that was how it discovered its own (fixed) IP address.

Cheers,
Wol

Change address on every DHCP renewal

farnz — Wed, 27 Aug 2025 14:43:40 +0000

The problem statement is that the old DHCP server failed. A new one was shipped in, with configuration but not persistent state restored from their VCS; by the time the new DHCP server arrived, all devices had lost their IP address.

When the new DHCP server started up, several systems failed to operate because they had hard-coded IP addresses that were not in config, the time taken meant the client had lost its IP (and was asking for a new DHCPOFFER instead of using DHCPREQUEST to renew the same lease), and the hardware failure meant that the leases database was lost.

How do you transform this into a state where the users discover within a human-reasonable (24 hours or so) timeframe that they'd hardcoded an address without making sure config matches? The two (broad strokes) solutions above are:

Ensure the address changes anyway (by various means) unless it's in config as a MAC→IP mapping, and make appropriate tradeoffs around connections blipping when the address changes.
Don't allow devices to get an IP unless it's in config as a MAC→IP mapping, so that hardcoding is safe.

If you've got another solution, I'm interested, because this is a hard problem to solve.

Change address on every DHCP renewal

paulj — Wed, 27 Aug 2025 13:39:17 +0000

It's just a text file of leases. Now, on OpenWRT, you're not actually configuring dnsmasq directly - you configure the OpenWRT UCI system, and when OpenWRT starts dnsmasq it creates the dnsmasq configs on the fly from the authoritatitive UCI information. Everytime dnsmasq is started, it has a freshly created lease file.

I don't understand the problem you envisage. If dnsmasq knows MAC addr X should get IP Y, it will give out IP Y if MAC X makes a request or a renew. If dnsmasq does not know (cause config was lost or corrupt) that X -> Y, then when MAC X asks, it will get some other IP. I've never had the problem you describe. Closest I've seen is that I've changed a wifi or NIC card/usb stick, and my hosts didn't get its usual static IP - just some random other one from the same pool. Until I update the OpenWRT config with the new MAC (openwrt automatically updates and restarts dnsmasq), and re-join/restart the network on the host.

Something similar can happen with IPv6 DHCP (which is not dnsmasq on OpenWRT, but a different dhcpv6 binary) with no hardware change, because in DHCPv6 static assignments are usually done via the DUID (DHCP UID), not the MAC. And the DUID sometimes can change on hosts (e.g. reinstall), which can be annoying. Still, you just end up with a different address than the static one - not "no address". So v6 the static host suffix (note: note the full IP - the single DHCPv6 suffix can be combined with and sued for multiple on-link prefixes!) survives swapping the network hardware, but not a reinstall on the same hardware.

Short lived certificate

cortana — Wed, 27 Aug 2025 11:35:25 +0000

A shame there isn't a bit in the prefix information (or maybe a DHCPv6 option, I dunno) to tell clients whether privacy addressing should be enabled or not...

Change address on every DHCP renewal

farnz — Wed, 27 Aug 2025 10:08:47 +0000

Right, but the only way to prevent that being a problem is to not have the second half at all - either you have a manually allocated address, or you're not on the network.

Otherwise, we hit the same problem; you have devices that are dynamically assigned addresses, where downtime causes them to move, but the addresses are hardcoded in places you don't know about.

Change address on every DHCP renewal

Wol — Wed, 27 Aug 2025 10:05:06 +0000

On my home router, I split my (RFC1918) address space into two. Any server-type devices I manually allocate an address in the first half, and put those addresses in hosts. Any unrecognised macs get allocated (randomly) from the second half. All devices are configured to get their IP via DHCP, but because the DHCPd knows which are static, they always get the same address.

(When I started doing this, I didn't understand how to set up dynamic DNS, and this for my home network was so much simpler.)

As I understand it, what happened at my site was they didn't put the server type stuff into the static half, and relied on the fact that when leases are renewed the IP doesn't change ... except because the down-time was longer than the ttl, the leases weren't renewed, they were re-allocated. And because the IPs were in the hosts file ... whoops!

Cheers,
Wol

Change address on every DHCP renewal

farnz — Wed, 27 Aug 2025 09:51:11 +0000

I don't see how to configure dnsmasq such that having assigned an address from a pool, it will maintain that device/address link forever, even in the face of the leases database being corrupted (which, from the sounds of things, is part of what happened at WoL's site). In particular, this means that once an address has been assigned, it's always assigned to that device (even if the pool is exhausted, and even if another device explicitly requests that address), and it's never going to assign a different address to that device (even if the leases database is corrupt, and even if the device requests a different address).

Perhaps you can clarify how your DHCP server of choice handles this.

Change address on every DHCP renewal

paulj — Wed, 27 Aug 2025 09:47:01 +0000

DHCP servers can assign static addresses from a wider dynamic pool (e.g., dnsmasq, used on openwrt and by libvirt, can). Perhaps the DHCP server you're used to is just a bit limited.

Linux distributions

Wol — Wed, 27 Aug 2025 08:23:47 +0000

> like systemd standardized on Debian's /etc/hostname over other distribution's equivalents

I'm pretty certain that convention pre-dates Debian, heck it pre-dates linux.

I remember using it on some Unix that was a bastardised SysV/BSD.

Cheers,
Wol

Short lived certificate

Cyberax — Wed, 27 Aug 2025 04:14:42 +0000

Unfortunately, that's not the case. ULAs are de-prioritized because of this: https://www.ietf.org/proceedings/113/slides/slides-113-6m...

The RFC specifies that GUA-GUA is preferred, even when ULA-ULA is available: https://www.ietf.org/archive/id/draft-ietf-6man-rfc6724-u...

Short lived certificate

ATLief — Wed, 27 Aug 2025 03:32:30 +0000

> It's very much possible for a device to prefer a public IPv6 source address even when talking to _other_ ULA addresses.

Most Linux systems automatically use the source address with the largest common prefix to the destination address. The trouble is if your DNS records return both a ULA address and a publicly-routable address, because the selection of the of the destination address is just based on round-robin.

Linux distributions

cesarb — Wed, 27 Aug 2025 01:23:11 +0000

> Ellie Kanning, who started the discussion, filed follow-up bug reports with OpenSSL, NetworkManager, and Freedesktop's desktop specifications, but none of the projects thought that maintaining a system-wide CRL was in-scope for them.

To me, that sounds like the sort of job traditionally done by Linux distributions. A Linux distribution can invent a shared mechanism, and patch all relevant software it distributes to conform to that. For instance, Debian invented a shared menu system, to which all applications distributed by Debian added menu entries, and all desktop environments distributed by Debian were patched to display these menu entries.

Once it's proven to work, it can be copied by other distributions (like RPM-based distributions copied the Debian "alternatives" mechanism, even though it was originally closely tied to Debian's package manager), and even standardized (like systemd standardized on Debian's /etc/hostname over other distribution's equivalents).

Short lived certificate

eythian — Tue, 26 Aug 2025 20:00:21 +0000

They shouldn't. They should have a fixed address, as you described, that can be used for incoming connections. Then besides that, they should also have the "privacy" address that rotates that is used for outgoing connections.

This said, when I've set up Linux servers on my network, privacy extensions are disabled by default, but when I connect a desktop, they're enabled. I'm not sure what does this switch.

There may be exceptions, I've been learning IPv6 lately but it's still not up to my decades of absorbed IPv4 knowledge.