LWN: Comments on "Fixing getrandom()"

Fixing getrandom()

Lawless-M — Fri, 19 Nov 2021 16:51:50 +0000

My VMs do not have cameras, radios or microphones

Fixing getrandom()

nix — Thu, 10 Oct 2019 20:28:24 +0000

Entropy draining is still useful, but not for blocking reads -- more, so that expensive methods of *accumulating* entropy only need to run when people have actually been reading from /dev/random in the first place (right now, adding entropy to a full pool will block until the pool drains a bit). These expensive methods are not theoretical: I have one attached to the machine I'm typing this on right now. Adding entropy non-stop will max out a core...

Fixing getrandom()

Jandar — Fri, 04 Oct 2019 11:36:02 +0000

The devices in question aren't installed individually. One image of a single install is put on thousands or millions of them, so generating host keys at installation time is the worst thing to do.

Fixing getrandom()

zdzichu — Fri, 04 Oct 2019 09:23:13 +0000

Because host keys need to be unique. Installer is used once to create a template. This template is then used number of times to create virtual machines.
Template cannot contain pregenerated host keys, because every VM would have the same key.
Using installer everytime when creating new VM is not feasible, installation process takes too much time. Creating new VM is something that should take no more than few seconds.

Fixing getrandom()

kmeyer — Fri, 04 Oct 2019 07:31:23 +0000

I don't see what possible benefit removing kernel entropy collection might offer. Many kernel entropy sources simply aren't available to userspace. Additionally, the kernel is the logical place to pool entropy for the system; it outlives all ephemeral userspace programs.

As to the rest, the APIs you ask for already exist.

> make getrandom() (and /dev/{u,}random) return an error if they have no randomness to provide.

getrandom(1, GRND_NONBLOCK) ⇒ -1/EAGAIN; poll(/dev/random, POLLIN, 0) ⇒ 0.

> This would mean that a userspace that doesn't initialize randomness early enough will just fail, loudly and deterministically.

All you need to do is add one of the above checks and a printf() to your userspace init process to produce the loud warning.

> So even the folks who try to “optimize boot time” by just removing boot-time items without thinking won't be able to build a broken system that boots but isn't secure.

That is the status quo with correct use of getrandom().

Fixing getrandom()

kmeyer — Fri, 04 Oct 2019 07:21:22 +0000

It doesn't fix the issue for live systems (are there many of those without RDRAND?), but for installed initial-boot systems: why not have the installer write out a random seed and optionally also sshd host keys?

Fixing getrandom()

kmeyer — Fri, 04 Oct 2019 07:14:40 +0000

1. The installation process generates a lot of IO, sufficient to seed the CSPRNG. It should emit a seed that can be used by the next boot (much like any other reboot entropy save).
2. Optionally, the installer can also generate and write out sshd host keys. There's not a lot of reason to wait until first boot for that.

Fixing getrandom()

kmeyer — Fri, 04 Oct 2019 06:50:55 +0000

If you have a device no one can use, why have it? So I'd say it's the design of Linux /dev/random that's at fault. Linux could drop their "entropy draining" concept tomorrow and have a /dev/random like BSD /dev/[u]random and everyone would be just as happy. There is no academic or practical basis in the "entropy draining" model Linux's /dev/random espouses.

Fixing getrandom()

NRArnot — Thu, 03 Oct 2019 10:40:36 +0000

As long as you can crank up the gain high enough to input "hiss" or "static" (electronic shot noise or cosmic background radiation) then even a microphone jack with no microphone plugged in, or a radio with no aerial, is a source of physical randomness. I don't know if the common on-board microphone sockets in the PC world can be used this way. I guess the problem for the truly paranoid, is how to tell whether what is coming in has been deviously compromised so as to only look random.

Personally I'd go with a boot parameter "paranoia = n" (maybe the current and maximum value is 11, with a nod to Spinal Tap). 10 would allow use of the random number generator on the CPU chip if there is one, and thereby solve all the problems other than the possibility that (insert conspiracy theory here).

Fixing getrandom()

mangix — Wed, 02 Oct 2019 00:54:39 +0000

FWIW OpenWrt master (and 19.07) uses this. It's faster than haveged.

Fixing getrandom()

kmeyer — Tue, 01 Oct 2019 20:54:43 +0000

Trusting the BIOS to provide good entropy seems like more of a mistake than jitter-entropy.

Fixing getrandom()

wahern — Tue, 01 Oct 2019 20:23:19 +0000

Also, just to be clear, the context for the boot seed was systemd. The overlap of embedded systems lacking both hardware entropy such as RDRAND and a reliable system clock but still running systemd is likely not very large. But then you also need to discount that by the odds of consecutive boots where systemd couldn't re-save a seed. *And* you need to discount it further by the odds the system was doing anything security critical. *And* you need to discount this by the odds that such a scenario would be distinguishable and exploitable.

Can this scenario exist? Sure. Does it exist? We should assume so. The only question is what's the risk, and does that risk outweigh the risk of not improving other aspects of the system's randomness semantics with the consequence that software will attempt to compensate *poorly*. And, again, what's that relative risk within the context of embedded system + systemd - RNG - clock?

Fixing getrandom()

wahern — Tue, 01 Oct 2019 19:07:53 +0000

I am assuming it, but I think it's a reasonable assumption--that there are adequate sources available to minimize the chance of a repeating nonce. The system clock was just an example. And of course there are systems where the assumption can't hold at all. So what? How many systems do there need to be to hold back everybody else? 1%? 0.1?%? 0.01%? 0%?

Some systems are just hopelessly broken when it comes to entropy. And that can't be fixed. But those systems are increasingly (and at this point likely *entirely*), small, embedded systems. It was always the responsibility of the designers of those systems to either make sure there's an entropy source available or design their firmware so that it wasn't necessary (i.e. no sshd generating a new private key on first boot). Are we going to let them hold back the inevitable *forever*? At some point we have to hold the stragglers' feet to the fire and cut our losses on the installed base--most of which would never upgrade, anyhow, and are unlikely to even be using getrandom(2) in the first place.

With the prevalence of not only RDRAND and similar on-chip sources, but also many other sources (e.g. Intel QuickAssist provided a hardware generator on the NIC controller since *before* it was even branded QuickAssist, EFI provides randomness, which in some cases comes from a hardware source--but that's a QoI issue), it's time to make the switch over to assuming (*loudly* assuming) that strong entropy is available at boot or will be available very shortly after boot (see CPU jitter hack as a last-ditch effort). Almost all of userland already makes this assumption, and has for quite some time, rightly or wrongly; now the ball is in the kernel's court to make good on that assumption to the best of its ability.

This *will* happen eventually, the only question is how long we'll wring our hands over misplaced concern for embedded platforms that are and were fundamentally broken. It's been almost 15 years since the VIA C3 included an on-chip RNG. Embedded designers have had ample warning about the necessity of providing strong entropy for a long time.

Fixing getrandom()

ceplm — Tue, 01 Oct 2019 19:02:22 +0000

Hold on! I haven’t said it cannot be real problem (although, couldn’t that first start of ssh daemon be a bit dealyed after whole system is up; systemd should be able to do something like that, shoudln’t it?). I just said a) there is something wrong with getrandom() defaults to the blocking version (sshd is certainly minority here, so it should be calling getrandom() with some specific options to make sure it gets lovely perfect random number), b) there is something wrong with gdm, when it doesn’t expect to be run rather early with possibly not enough entropy for something which really doesn’t require super-random numbers (it should certainly call getrandom(0) because it really doesn’t matter that much how superspecial those random numbers are). c) well, those IoT devices should know about this situation and somehow collect more enthropy themselves (mixing in MAC addresses, some their serial numbers; yes, it is predictable for the given device, but possibly less predictable for anonymous device in general? Not sure.)

Fixing getrandom()

alonz — Tue, 01 Oct 2019 17:28:29 +0000

You're assuming that there is a reasonably-initialized system clock at the point where entropy is required – this is just as wrong as any of the other assumptions regarding entropy.

Fixing getrandom()

zlynx — Tue, 01 Oct 2019 17:05:19 +0000

What do you do about the fact that IoT devices built with identical hardware and identical software boot up and produce one of *maybe* 4,000 or so *identical* /dev/urandom streams. It's pretty much based on temperature which can *slightly* modify the interrupt rates vs clock cycles.

At any rate, it's a real problem. Because if they do something during boot such as generate their own SSH or SSL certificates the attacker only has to guess a few possibilities.

Fixing getrandom()

Cyberax — Tue, 01 Oct 2019 16:53:15 +0000

Using /dev/random is a bad idea in general. There's no reason it's more secure than /dev/urandom and it can lead to large delays.

Fixing getrandom()

ceplm — Tue, 01 Oct 2019 09:49:21 +0000

I was working on the mistaken assumption that getrandom() gets data from /dev/random. It doesn’t, it is actually desgined well, and this is just a bug, which need to be fixed, no more fuss about it.

Fixing getrandom()

ceplm — Tue, 01 Oct 2019 09:39:26 +0000

Fixing getrandom() is removing it, or providing getnormalrandom() (which reads random data from /dev/urandom without any other checks). In the moment when there is silly method with a nice kernel call and reasonable method which is even slightly more complicated to use, everybody uses the first one even in the situation when they shouldn’t. Data from /dev/random are not more random or somehow with a pixie dust on the top. They are exactly the same, but sometimes /dev/random blocks for reasons which don’t matter for almost anybody (https://www.2uo.de/myths-about-urandom/).

Fixing getrandom()

NYKevin — Tue, 01 Oct 2019 02:46:37 +0000

Well, if that's too early, that means you're plausibly running all kinds of nonsense, like Python.

Fixing getrandom()

wahern — Mon, 30 Sep 2019 19:30:53 +0000

Who cares if it's reused? The same 32-byte random seed cryptographically mixed with a non-repeating nonce, like the system clock, would have the same strength as a new 32-byte random seed, presuming the seed remains confidential. Long-term it's better to change the seed in case confidentiality was unknowingly violated or the nonce repeats (clock reset), but as a practical matter those aren't prerequisites to having a reasonably sane and secure system, especially considering the alternative--a multiplicity of userland CSPRNGs all scavenging for entropy independently.

Fixing getrandom()

Otus — Mon, 30 Sep 2019 13:19:11 +0000

> If things as late as GDM/gnome-session are still "early boot", then which service does not count as early boot? See the problem?

From the point of view of the random pools, before this change, anything before the user gets a login screen is early boot. That's when you start getting more than a trickle of entropy.

Fixing getrandom()

anton — Mon, 30 Sep 2019 12:49:23 +0000

Thermal noise is relatively small compared to photon noise if the sensor receives significant photons, but may be enough for initializing the RNG. And of course you don't want to be have so much brightness and so much exposure that the sensor saturates, but you can recognize anything approaching saturation, and then use shorter exposure time, if too many pixels are saturated. Combining high gain with long exposure will give more thermal noise in darkness, but produce saturation if there is light.

Fixing getrandom()

excors — Mon, 30 Sep 2019 11:59:57 +0000

I think you can get a decent amount of entropy even if there is no light on the camera, because of thermal noise in the sensor. That's probably safer than receiving significant light, because if the sensor gets saturated then you'll get a pure white image with no entropy. Phone cameras don't have physical shutters but the sensors often have some non-exposed pixels around the edge (for black level calibration etc) and you could probably use those.

> you need a way to get the raw data (transformation into JPEG usually tries to get rid of the noise that we want for the RNG)

It's not just the JPEG compression - the Android camera API is happy to give you uncompressed YUV but that still wouldn't be raw enough. You'd want the (typically) 10-bit Bayer data directly from the sensor, before the ISP has tried to make it look pretty (doing noise reduction, adjusting levels in a way that might saturate the noise out of existence, smoothing the image, etc). And you probably want to manually configure the sensor to maximise noise (long exposure, high gain, disable binning, etc). Android provides enough control to let applications request that, but I don't know how many of the camera drivers implement it fully, so it's probably not a very portable approach.

Fixing getrandom()

patrakov — Mon, 30 Sep 2019 11:57:03 +0000

Yes, but with a few phrases in the commit message that I not necessarily agree with (or maybe should interpret as sarcasm, because then it makes perfect sense). Let me quote the problematic sentence.

"""
While this was triggered by what is arguably a user space bug with GDM/gnome-session asking for secure randomness during early boot, when they didn't even need any such truly secure thing, the issue ends up being that our "getrandom()" interface is prone to that kind of confusion, because people don't think very hard about whether they want to block for sufficient amounts of entropy.
"""

If things as late as GDM/gnome-session are still "early boot", then which service does not count as early boot? See the problem?

Fixing getrandom()

joib — Mon, 30 Sep 2019 10:54:10 +0000

Seems this approach was merged:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/...

Fixing getrandom()

anton — Mon, 30 Sep 2019 08:26:09 +0000

I do not know if a microphone or radio are good random sources, but a camera is. The resolution of camera sensors is high enough that the randomness of the photons coming in is reflected in the raw sensor output (and it is a lot for a (not too) bright picture). However, that means that the sensor must be on and receive significant light on booting, and you need a way to get the raw data (transformation into JPEG usually tries to get rid of the noise that we want for the RNG).

Fixing getrandom()

patrakov — Sun, 29 Sep 2019 20:27:49 +0000

This is exactly why I say "don't". Too many bugs stem from our desire do achieve the impossible instead of giving up immediately.

OTOH, jitter entropy will definitely help here, up to the point of making it completely unneeded to save entropy between reboots.

Fixing getrandom()

NYKevin — Sun, 29 Sep 2019 20:05:43 +0000

I recommend clicking through to that bug report. This is more complicated than I had imagined, because in cases where people take images of live systems, you really shouldn't credit any "stored" entropy at all (because it's been duplicated umpteen times into other instances of the same image, so it's no longer unpredictable). But you can't know that someone imaged the system, so how do you square that circle?

Fixing getrandom()

flussence — Sun, 29 Sep 2019 19:37:11 +0000

So, I had a lingering question that the last paragraph of that page answers sufficiently.

I've got a system where the NVRAM is probably fine, but it has a broken EFI implementation (AMI), where nobody bothered to implement deallocating deleted vars, so eventually it'd start returning -ENOSPC for every write operation. Me naively leaving pstore panic logging enabled soon flushed that out (followed by real panic at efibootmgr failing, and a day of downtime trying to figure out what went wrong and tearing the room up to get at a CMOS jumper).

The kernel help text for EFI features could use a gentle reminder that yes, EFI firmware *is* written by the same nincompoops as the bad old BIOSes of the 90s, and should be equally mistrusted.

Fixing getrandom()

mezcalero — Sun, 29 Sep 2019 09:26:52 +0000

systemd reads a random seed off disk just fine for you, no need to write any script for that. Problem with the approach is that it's waaaay too late: we can only credit a random seed read from disk when we can also update it on disk, so that it is never reused. This means /var needs to be writable, which is really later during boot, long after we already needed entropy, and long after the initrd.

Hence, no, systemd is not causing this, systemd does what it can, but it can't magically create entropy where there is none.

Or to say this differently: that "arbitrary shell script" you are envisioning, what is it supposed to do? Where would it derive entropy from where neither the kernel nor systemd do or could do it at least as good?

if you care, have a look here, about the approach systemd takes to help you with the general problem: https://systemd.io/RANDOM_SEEDS.html

Lennart

Fixing getrandom()

corsac — Sun, 29 Sep 2019 09:01:34 +0000

Factorable.net problem is not only for systems not restoring random seed. Most of them are devices booting for the first time and generating long term keys (ssh etc.) with a really similar entropy state (because of unseeded RNG). It's likely there are a *lot* of devices like these beeing shipped every day, even more so with cloud VMs, so breaking those is a really bad idea I think.

Fixing getrandom()

patrakov — Sun, 29 Sep 2019 07:29:48 +0000

The story has received an update: Linus Torvalds posted a patch to get the entropy from the timing of schedule() calls. Very similar to jitter entropy.

https://lore.kernel.org/lkml/CAHk-=wgjC01UaoV35PZvGPnrQ81...

Fixing getrandom()

edeloget — Sun, 29 Sep 2019 06:04:58 +0000

Well, for the past ten years I have been developing specialized distributions for network devices, and they tend to not have any of the above. And yes, SSH keys generated on the first boot are really a pain (actually less of a pain these days because newer CPUs tend to propose a hwrng but hey, this is an industry where you are routinely dealing with severly out-of-date CPUs...).

Fixing getrandom()

joib — Sat, 28 Sep 2019 20:38:13 +0000

Probably the BSI cares only about x86(-64) Windows on cpu's supported by currently maintained Windows versions, so they can assume the presence of TSC? So it doesn't really address what seemed to be Linus main objection.

(Though in my non-expert opinion, it seems having a jitter entropy generator in the kernel for supported targets would be the least bad approach of those discussed here. Those few that run unsupported targets are hopefully sufficiently clueful that they can use a hw RNG, haveged, or maybe they don't need early boot random numbers anyway.)

Fixing getrandom()

patrakov — Sat, 28 Sep 2019 19:52:49 +0000

Could you please submit some documental evidence of this approval? Asking because it would be the perfect response to the last paragraph in https://lore.kernel.org/lkml/CAHk-=whz7Okts01ygAP6GZWBvCV...

Fixing getrandom()

patrakov — Sat, 28 Sep 2019 19:44:28 +0000

This site was indeed mentioned in the discussion. However, a return of the factorable.net issue for systems that do not properly restore the random seed is preferable, in my viewpoint, to breaking almost all systems.

Fixing getrandom()

mads — Sat, 28 Sep 2019 16:36:52 +0000

But it would be trivial to add that to systemd too, and why does it have to be a shell script? (not that systemd can't use bash, but why should it)

Fixing getrandom()

corsac — Sat, 28 Sep 2019 12:07:19 +0000

Agreed. I didn't read the whole thread, but I'm surprised not to see any reference to “Mining your Ps and Qs” from Usenix Security 2012 (https://www.usenix.org/conference/usenixsecurity12/techni... and https://factorable.net/weakkeys12.extended.pdf)

Fixing getrandom()

dd9jn — Sat, 28 Sep 2019 09:57:57 +0000

So in Libgcrypt we have been urged to move to getrandom to avoid some issues in the early boot phase and to get better performance in almost all cases. Weakening getrandowm now is a no-go and would badly reflect on the security consciousness of _some_ kernel folks.

Using Stephan Müller's jitter based entropy generator inside the kernel is by any means the Right Thing to do - even if it is for now only a fallback. In Libgcrypt's Windows version we already use it because on Windows the JitterRNG is the only non-external-hardware RNG which has been approved by Germany's BSI for use in restricted communication at the VS-NfD level. On Linux getrandom has been evaluated as fine but nevertheless we mix some entropy from the JitterRNG into our own entropy pool. Right, we also use RDRAND in addition and that is technically okay. But because RDRAND can't be evaluated the evaluation of Libgcrypt assumes that RDRAND adds 0 bits of entropy to the pool.