LWN.net Logo

On the safety of Linux random numbers

Random number generation is an important operating system function. The generation of networking sequence numbers, cryptographic session keys, and public keys all depend on the creation of numbers which are sufficiently random that they cannot be guessed by an attacker. Weak random numbers can lead to session hijacking, disclosed secrets, forged identities, and predictable umber hulks. Any system which is serious about security has to be serious about creating good random numbers.

Doing that, however, can be a challenge for computers. As a general rule, designers of computers like to make hardware which does the same thing every time. Randomness is not normally a desirable feature in computer operation; for most systems, it is restricted to emacs responding to mistaken keystrokes. So, while there is no shortage of algorithms which can produce a random-seeming sequence of numbers, those numbers are not truly random. Restart the algorithm with the same initial conditions, and the same sequence of numbers will result.

Linux implements a purely algorithmic random number generator, accessible as /dev/urandom. Its results are good enough for most purposes, but there are times when true randomness is needed. To that end, the kernel attempts to harvest randomness (called "entropy") from its environment. The timing between the keystrokes as your editor types this article, for example, exhibits some randomness. The same is true of, for example, the timing of disk interrupts. The lower bits of the system time stamp counter can also provide a bit of entropy. The kernel collects this entropy into a special pool of bits, and uses this entropy pool when true random numbers (obtained from /dev/random) are required. The amount of accumulated entropy is also tracked; if there is insufficient entropy in the pool to satisfy a random number request, the requesting process will block until the needed entropy arrives.

One of the most common ways of putting entropy into the pool is to register interrupt handlers with the SA_SAMPLE_RANDOM flag. That flag tells the kernel that the indicated interrupt will arrive at random times, so its timing can be used to generate entropy. This interface has been in place for many years, but Matt Mackall has recently decided that it is not the best way to go. So he has posted a series of patches removing SA_SAMPLE_RANDOM from a large number of request_irq() calls. Most of the changes are not controversial. For example, a number of disk drivers set SA_SAMPLE_RANDOM, but also use the block-specific add_disk_randomness() function. Removing SA_SAMPLE_RANDOM in those cases eliminates a source of redundant "entropy." But Matt rekindled an old debate when one of his patches removed SA_SAMPLE_RANDOM from a set of network drivers.

The issue with network drivers is this: network interrupts are created by incoming and outgoing packets. If an attacker gets access to the network segment used by a target system, that attacker can observe the timing of packets entering and leaving that system. The attacker can also influence that timing by generating packets and sending them to the target in a carefully-timed manner. Over the years, a number of people have worried that a well-connected attacker might be able to guess the contents of the entropy pool and predict future random numbers.

Others argue that nobody has shown a scenario where the ability to observe and generate packet timings could actually lead to the compromise of the entropy pool. The actual timing of packets hitting a given system can only be reliably observed by another system on the same network segment. But network segments are almost never shared anymore; most systems tend to be plugged into switches, and a switch will hide packets and change their timing. In addition, anybody who is in a position to get onto a target system's network segment is quite likely to be able to obtain physical access to the target itself. At that point, the installation of a keystroke logger or hostile kernel patch seems easier than trying to guess where the entropy pool will go.

If we assume a particularly determined and masochistic attacker, however, then we can start to think about the other challenges this person will have to face. One is guessing the contents of the entropy pool at a given time. Such a guess will have to be made by observing the random numbers generated by the system, which can be done by looking at sequence numbers and keys emitted by that system. Then the attacker will have to find a way to reverse the algorithm (SHA-1) which is used to generate a given random number from the pool. That reversal will generate a large set of possible pool values which could all hash to the same value, so the attacker must be prepared to work with many simultaneous possibilities.

Once the pool has been guessed, it is time to predict its future value, as determined by the incoming entropy. The problem here is that the timing of packets on the wire does not exactly match the timing of interrupts within the kernel. There are delays within the network card, delays in DMAing a packet into main memory (which can be influenced by other memory traffic being generated in the system), variable interrupt handling times caused by critical sections which mask interrupts, cache misses, etc. Then there is the occasional mixing of bits from the time stamp counter, the value of which is not available to the attacker. All told, it is a fair stretch to go from an observation of traffic on the network to any sort of guess as to what the random number generator will produce next.

Meanwhile, many systems running as network servers have access to relatively few sources of entropy. If interrupt timings from network interfaces are made unavailable, those systems could run out of entropy altogether. Given that need, and given that most developers seem unworried about the potential weaknesses, the use of network timings is unlikely to go away anytime soon. What might happen, however, is the addition of some sort of runtime configuration option. Truly paranoid administrators could then disallow entropy from network interfaces. Those who are merely worried could, instead, use those timings, but reduce the amount of entropy which is credited to a network interface timing value. And most of the rest of us will probably leave things the way they are now.

[See also: this paper by Z. Gutterman, B. Pinkas and T. Reinman [PDF] on potential weaknesses in the Linux random number generator (thanks to Neil Harris).]


(Log in to post comments)

On the safety of Linux random numbers

Posted May 11, 2006 5:02 UTC (Thu) by Thalience (subscriber, #4217) [Link]

It surprises me that more computers don't have hardware random number generators. At least "server class" hardware anyway.

Do USB interrupts have the right hooks to add entropy? Perhaps a simple device to send random USB HID events on a server could be developed.

Why not get randomness from sensors?

Posted May 11, 2006 5:48 UTC (Thu) by Ross (subscriber, #4065) [Link]

There are some untapped sources of entropy: the hardware sensors on most motherboards. The temperature, voltage, fan speed, etc. sensors could be manipulated by someone with physical access, but only to a limited precision. Doing some kind of differential sampling and not counting zero inputs as having any entropy should help.

Is there a reason the drivers for those devices don't contribute to the entropy pool?

Why not get randomness from sensors?

Posted May 11, 2006 6:08 UTC (Thu) by jwb (guest, #15467) [Link]

It is fantastically expensive to communicate with the most common sensor hardware. Often the system has to bang bits over a two-wire port. It is reasonable to access these devices once every second or so, but I doubt that you'd be able to sample them often enough to generate substantial entropy.

It would be nice if hardware entropy devices were more common. There are plenty of random processes out there in the electrical world, like shot noise.

Why not get randomness from sensors?

Posted May 11, 2006 14:44 UTC (Thu) by pjones (subscriber, #31722) [Link]

I think you're wrong about what "substantial" means here. It doesn't need to be enough entropy to use as the system's only source. It needs to be enough to pervert the data from all the other sources in a way that masks their (potential) weeknesses. That requires surprisingly little data, if it is truly unavailable to attackers.

To that end, the bigger worry here is that it's just the sort of data you might want to stick in SNMP for your monitoring infrastructure to check on.

Why not get randomness from sensors?

Posted May 11, 2006 21:54 UTC (Thu) by giraffedata (subscriber, #1954) [Link]

It doesn't need to be enough entropy to use as the system's only source. It needs to be enough to pervert the data from all the other sources in a way that masks their (potential) weeknesses.

You seem to be describing a means of creating entropy out of nothing. If all the other sources provide 1000 bits per second of entropy and the hardware sensor gives you another 10 bits per second, you've got at most 1010 bits per second of entropy no matter what you do with those 10 new bits.

So I think "substantial" is the same amount no matter how you look at it.

Actually, I think the sensors mentioned have negligible entropy to contribute. You read them all digitally, and given one reading, you can predict very well what the reading will be a second later, to the full precision of the sensor.

Intel i8x0

Posted May 11, 2006 6:40 UTC (Thu) by ncm (subscriber, #165) [Link]

As I understand it, Intel included an interface for getting efficient access to truly random numbers in their chipsets starting at i810 or so. In the early versions, this was a separate, optional chip wired to a dedicated pin. Of course few motherboard manufacturers left pads for it, and even fewer built boards with the chip present. The presence or absence of a random-number generator chip is not high on the list of motherboard features that early adopters (i.e. gamers) look for. Intel marketing must have interpreted this as an entire lack of interest among users, and so omitted the (very cheap!) feature as they integrated the various outboard chips.

So, it appears we can blame Intel marketing for sabotaging this solemnly promised feature of all future Intel chipsets. As promised, all are equipped to report whether they can produce random numbers; they all say "no".

(This is the best reconstruction of events I have been able to establish through Google searches. Someone else may have better information not readily googled. I welcome corrections.)

Hardware RNGs in chipsets and CPUs

Posted May 11, 2006 14:48 UTC (Thu) by hmh (subscriber, #3838) [Link]

It is more complicated than this. Intel placed the HRNG inside its FWH (firmware hub). I.e, inside a FLASH memory device that is supposed to host the BIOS. Were it inside the MCH (the north bridge), all machines would have it and this story could be very different indeed.

The Intel FWH HRNG is very slow, but it appears to be of very high quality... Unfortunately, the FWH was quickly made an *optional* component of the chipset for whichever reason, and that effectively killed the whole idea. Sometime after that, Intel declared the whole "more secure computers by using an Intel chipset with a HRNG" idea a bust and stopped even caring about producing FWHs with HRNGs.

After that blow, often not even Intel itself would uses their FWH. Take a Intel D875PBZ motherboard for example. I have one, and direct access to three others. Two of them have Intel FWHs, of which one has a working HRNG and the other does not (the HRNG is disabled on silicon). The two other boards use compatible FWHs from other chip manufacturers, that don't have a HRNG either.

Add to it that (AFAIK at least) MS Windows does not have a common interface to get the random numbers from (Unix is easy, provide them through /dev/u?random and everybody uses it), and nobody was really paying much attention to the Intel device driver required to get the data from the FWH...

Now, VIA did things almost right. They placed an *extremely* fast HRNG inside their Nehemia CPU cores (but last time I checked, you'd have to talk directly to them if you wanted to make sure a batch of Nehemia CPUs would come with enabled cores: they disabled the HRNGs when they failed the factory test, instead of scrapping the CPU), added a good hardware crypto engine, and made a major marketing party out of it. Not happy with just one, the newest VIA cores have two HRNGs in different areas of the chip... so you get double the bandwidth, and somewhat less correlation on the output stream.

A heavily modifed version of rng-tools got about 2Mbit/s of random bits from such a Nehemiah CPU (at its highest quality mode, at lowest quality, it is probably on the 12 Mbit/s range in a dual HRNG CPU). This work was sponsored by mekensleep.com, and is available in Debian experimental under the GPL license. One can also use Martin Peck's modified hw_random linux module if they prefer a kernelspace solution.

Intel i8x0

Posted May 11, 2006 21:42 UTC (Thu) by giraffedata (subscriber, #1954) [Link]

Intel included an interface for getting efficient access to truly random numbers in their chipsets starting at i810 or so

How does it generate truly random numbers?

Intel i8x0

Posted May 12, 2006 7:33 UTC (Fri) by ncm (subscriber, #165) [Link]

How does it generate truly random numbers?

Physics.

There are plenty of ways to extract truly-random noise from the detailed behavior of electronic components. Generally these sort out into those that go below the statistical averaging of "electric current" to look at thermal variation of the motion of individual electrons ("shot noise"), and those that depend on quantum indeterminacy. Given random analog noise, whether as a current, voltage, or timing noise, it's not hard to turn it into unbiased numbers.

On the safety of Linux random numbers

Posted May 11, 2006 7:17 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link]

you are forgetting that there is a huge number of embeded systems that are lacking external inputs (and drives in many cases), but still need entropy.

these are hardly 'server class' as I think you are defining it.

On the safety of Linux random numbers

Posted May 11, 2006 17:11 UTC (Thu) by drosser (guest, #29597) [Link]

Even the mighty Mainframe has had its share of comeuppances in regard to random numbers. And when you find a bug in hardware, it's really difficult and costly to fix.

Reminds me of a quote I saw somewhere

Posted May 11, 2006 6:43 UTC (Thu) by toreanderson (guest, #14736) [Link]

«Random number generation is too important to be left to chance» :-)

Reminds me of a quote I saw somewhere

Posted May 11, 2006 22:18 UTC (Thu) by nix (subscriber, #2304) [Link]

Wikipedia says this is due to Robert Coveyou.

Myself I prefer Knuth's line (from the introduction to TAOCP chapter 3) that (in italics) `random numbers should not be generated with a method chosen at random'. He then goes on to provide several examples of awful PRNGs, such as the original rand(), and shows why they're bad...

(... and a few pages later, whose name pops up, but Robert Coveyou!)

/dev/urandom

Posted May 11, 2006 7:37 UTC (Thu) by ekj (guest, #1524) [Link]

Actually, I'm fairly certain urandom is *not* purely algoritmic "pseudo-random" numbers.

My impression was that urandom would give you real random numbers based on the collected entropy-pool, but that it (and this is the difference to /dev/random) will *not* block when entropy gets "too low", but instead continue with pseudo-random numbers.

Did I misunderstand this ? Or did our editor misunderstand it ?

/dev/urandom

Posted May 11, 2006 9:38 UTC (Thu) by kleptog (subscriber, #1183) [Link]

Your understanding matches mine. I've seen reports about people complaining that programs consuming large amount of data from /dev/urandom cause programs using small amounts from /dev/random to block. On other systems the data comes from seperate pools whereas on Linux you get the same data from both, except /dev/urandom doesn't block but becomes PRNG when the pool is empty.

/dev/urandom

Posted May 11, 2006 20:35 UTC (Thu) by copsewood (subscriber, #199) [Link]

My understanding of this is that /dev/urandom reseeds a pseudo random sequence using values from /dev/random as seeds, with a reseeding frequency dependent upon how much genuine entropy is available.

On the safety of Linux random numbers

Posted May 11, 2006 9:49 UTC (Thu) by zooko (subscriber, #2589) [Link]

It's an unfortunate myth that /dev/random provides "true" randomness and /dev/urandom provides "pseudorandomness". A more accurate summary would be that they both provide pseudorandomness, but that /dev/random attempts to be safer by blocking when the amount of pseudorandomness that it would output would be greater than (some complicated and poorly understood magic estimate of) the amount of randomness that has been fed in.

In practice the blocking behavior of /dev/random causes more problems -- including more security problems -- than the unlimited output of /dev/urandom causes.

If it weren't for an unfortunate security flaw in /dev/urandom (concerning initialization during system startup), then there would be no good reason for anyone to use /dev/random.

On the safety of Linux random numbers

Posted May 11, 2006 17:21 UTC (Thu) by Ross (subscriber, #4065) [Link]

I don't see how /dev/random can be called pseudorandom. It uses a fixed algorithm to produce the output, sure, but the amount of output is no greater than the amount of physically random input. It's not just a seemingly-random number sequence with an unknown starting point.

Now /dev/urandom is also more than a pseudorandom number generator, but unlike /dev/random, it doesn't keep track of entropy, so it may dengenerate into a pseudorandom sequence at any time without warning.

On the safety of Linux random numbers

Posted May 11, 2006 17:47 UTC (Thu) by zooko (subscriber, #2589) [Link]

What you say is indeed the widespread belief, and the original motivation for the distinction between /dev/random and /dev/urandom, but the truth of this belief has not been proven.

For starters, there is no guarantee that /dev/random is outputting an amount no greater than the amount of random input. (Indeed, it is impossible to prove that it did without being able to read the mind of your attacker.) For example, the patches which form the subject of this LWN story are one attempt to tweak the estimate of the randomness input because someone (Matt Mackall) thought that the current estimator was estimating something incorrectly.

My point isn't that the estimator is particularly bad -- my point is that the existence of the estimator demonstrates that there is no hard guarantee that the amount of output is no greater than the amount of input.

In theory, it is an open question whether /dev/random or /dev/urandom is more secure against cryptanalysis. In practice, they are very likely both secure, except that the blocking behavior of /dev/random introduces different vulnerabilities. (And except for a few unfortunate flaws in /dev/urandom...).

On the safety of Linux random numbers

Posted May 11, 2006 18:44 UTC (Thu) by Ross (subscriber, #4065) [Link]

I see what you are saying. Yes, it is an open question just how closely the entropy estimates match the actual gathered entropy. Certainly the double-counting (like corrected in the floppy driver patch) doesn't make one feel secure. However the counts are in general very conservative so it is more likely they are an underestimate than an overestimate.

About comparing the security of /dev/random and /dev/urandom: I don't understand the problem. They are equivalent under cryptanalysis because they use exactly the same mechanism to produce output, or at least that was my understanding. The only difference, which is not cryptographic, is the blocking behavior of /dev/random. Blocking readers when the estimated entropy is too low can not make it easier to reverse the hashing or otherwise attempt to determine the contents of the entropy pool. Now it may not make it any harder, but that is a different statement. That possibility is probably acceptable because the only disadvantage to blocking is as you said earlier: denial of service and slowed performance. But the system where both methods are present is good because system administrators can use /dev/urandom (or even change the /dev entry) all the time if that is what they really want.

On the safety of Linux random numbers

Posted May 11, 2006 11:32 UTC (Thu) by kleptog (subscriber, #1183) [Link]

I always thought these worries about predicatable interrupts creating bad entropy to be a bit wierd. After all, if you have a cycle counter where you only look at the last two bits, the result will be unpredicable based on how many cycles the intruction took at the time the interrupt signal was generated.

Imagine my surprise when I look at the code and see they don't use the cycle counter on most architechtures, but base it on the number of *jiffies*, a counter which goes maybe 1000 times per second.

No wonder people are worried about predictability. On the scale of milliseconds a lot of things become predicatable. Maybe we should be doing something about that first. We have high-resolution timers in the kernel, do we not?

On the safety of Linux random numbers

Posted May 12, 2006 7:35 UTC (Fri) by ncm (subscriber, #165) [Link]

That astonishes me too. I always assumed they were using low bits of the cycle counter. It seems an easy patch. Any clue why it hasn't been done?

""Any one who considers arithmetical methods of producing random digits is, of course, in a state of sin."

Posted May 12, 2006 13:22 UTC (Fri) by Max.Hyre (subscriber, #1054) [Link]

John von Neumann :-)

Copyright © 2006, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds