Cryptanalysis of the random number generator of the Windows operating system

Abstract

The PseudoRandom Number Generator (PRNG) used by the Windows operating system is the most commonly used PRNG. The pseudorandomness of the output of this generator is crucial for the security of almost any application running in Windows. Nevertheless, its exact algorithm was never published. We examined the binary code of a distribution of Windows 2000. This investigation was done without any help from Microsoft. We reconstructed the algorithm used by the pseudorandom number generator (namely, the function CryptGenRandom). We analyzed the security of the algorithm and found a nontrivial attack: Given the internal state of the generator, the previous state can be computed in 2 23 steps. This attack on forward security demonstrates that the design of the generator is flawed, since it is well known how to prevent such attacks. After our analysis was published, Microsoft acknowledged that Windows XP is vulnerable to the same attack. We also analyzed the way in which the generator is used by the operating system and found that it amplifies the effect of the attack: The generator is run in user mode rather than in kernel mode; therefore, it is easy to access its state even without administrator privileges. The initial values of part of the state of the generator are not set explicitly, but rather are defined by whatever values are present on the stack when the generator is called. Furthermore, each process runs a different copy of the generator, and the state of the generator is refreshed with system-generated entropy only after generating 128KB of output for the process running it. The result of combining this observation with our attack is that learning a single state may reveal 128KB of the past and future output of the generator. The implication of these findings is that a buffer overflow attack or a similar attack can be used to learn a single state of the generator, which can then be used to predict all random values, such as SSL keys, used by a process in all its past and future operations. This attack is more severe and more efficient than known attacks in which an attacker can only learn SSL keys if it is controlling the attacked machine at the time the keys are used.