Comment by Taek

5 days ago

You don't need 256 bits of entropy, you only need 128.

I have tested this method on over 100 different CPUs and I have never seen such consistent output. I'm genuinely surprised to see that you only hit 92 bits of entropy, but that can trivially be fixed by doing 10x the iterations. 500 iterations is still going to put you under a millisecond of cost.

And, for what it's worth, code I've actually shipped has combined the above technique with Fortuna, and has typically targeted 2000 bits of entropy rather than 128 (for security buffer).

EDIT: I reviewed his code, and he's not hashing between calls to check the clock; the hash call itself causes the CPU to heat up in arbitrary ways which changes the timing between hashes and introduces more entropy; removing that call basically entirely defeats the idea behind the technique, these results are fully invalid.

---

I updated the code to insert the hash call, this is what I got for his original code on my machine, and the updated code with hashing on my machine (and the difference is cryptographically meaningful):

  === Original C — no hashing ===
  Clock resolution: 0.000000001
  Deltas (ns):   50   34   19   19   13   13   13   13   13   14   13   13   13   13   13   14   13   13   14   12   13   14   13   13   13   14   13   13   14   12   13   14   13   13   14   12   13   14   13   14   13   12   13   14   14   13   13   13   13
  Deltas of deltas:   -16  -15    0   -6    0    0    0    0    1   -1    0    0    0    0    1   -1    0    1   -2    1    1   -1    0    0    1   -1    0    1   -2    1    1   -1    0    1   -2    1    1   -1    1   -1   -1    1    1    0   -1    0    0    0
  Maximum entropy: 90

  === C with SHA-256 between clock reads ===
  Clock resolution: 0.000000001
  Deltas (ns): 756852 1287  542  470  472  445  442  436  434  439  488  435  433  434  440  439  439  435  432  433  435  432  433  433  429  433  453  441  437  437  431  433  432  430  431  438  436  434  431  433  435  436  435  433  430  436  435  437  428
  Deltas of deltas:  -755565 -745  -72    2  -27   -3   -6   -2    5   49  -53   -2    1    6   -1    0   -4   -3    1    2   -3    1    0   -4    4   20  -12   -4    0   -6    2   -1   -2    1    7   -2   -2   -3    2    2    1   -1   -2   -3    6   -1    2   -9
  Maximum entropy: 188

3 comments

Taek

sltkr 5 days ago

The increase in calculated entropy comes from the first iteration being slower than the rest, but that's a bit misleading, because the first call is always going to be slower.

Can you run the program 10 times and show me how much variance there actually is in the first column? Because if all the values lie between (say) 756000 and 757000 that's actually just 10 bits of entropy, not 19.5, and if the same applies to the other values, you're much closer to the original 90 bits.

Taek 5 days ago
I ran it 500,000 times, discarding the 10% most entropic results ... in the hopes of arriving at a relatively conservative estimate for the amount of entropy you actually get from each iteration. Here's the prompt I used to generate the code: https://chatgpt.com/share/6ab2df4a-7f94-83ea-aecf-1bb57c4838...
And here are the results of running that code:
=== No hashing === Clock resolution: 0.000000001 seconds Clock reads: 500,000 Second-difference outcomes: 499,998 Retained outcomes: 449,998 (90.000%) Average Shannon information: 1.755579 bits/retained outcome Marginal min-entropy estimate: 1.339460 bits/retained outcome Lag-1 conditional min-entropy: 0.960079 bits/retained adjacent outcome Conservative descriptive proxy: 0.960079 bits/retained outcome Proxy scaled per clock iteration: 0.864067 bits/iteration These are empirical timing statistics, not a proven entropy rate. === One SHA-256 between clock reads === Clock resolution: 0.000000001 seconds Clock reads: 500,000 Second-difference outcomes: 499,998 Retained outcomes: 449,998 (90.000%) Average Shannon information: 4.205076 bits/retained outcome Marginal min-entropy estimate: 3.610848 bits/retained outcome Lag-1 conditional min-entropy: 3.351217 bits/retained adjacent outcome Conservative descriptive proxy: 3.351217 bits/retained outcome Proxy scaled per clock iteration: 3.016082 bits/iteration These are empirical timing statistics, not a proven entropy rate.
------------
As GPT helpfully points out, this isn't a proven guarantee, but a reasonable estimate is somewhere between 3 and 4 bits of entropy per hash. That means 50 is actually enough, though if you want to be conservative I don't think there's any harm in doing 500 or even 5,000 instead of 50. And, if you are going to be using this in a hostile environment, it doesn't hurt to also add a fortuna-like accumulator that resets your entropy every once in a while.
I said this in another reply as well, but the reason that you get 3-4 bits of entropy per hash is because of the fundamental nature of CPUs. In addition to having considerable professional experience with cryptography, I also have considerable professional experience with hardware; hardware is fickle as hell, especially when your transistors are tens of nanometers large. Every time you flip a bit, you expend some energy, which heats up the chip, and the heat changes the timing of the next clock cycle. Chips are composed of literally billions of transistors, and each one is going to have a different temperature, because clock cycles last less than a nanosecond (well, embedded hardware is slower but the same idea still applies reliably) and that's not enough time for temperature deltas to dissipate across the chip.
Hashing is particularly chaotic because it lights up a different set of transistors on each clock cycle, which means the hotspots on the chip are being jerked around. Some transistors are going to light up 5-10 times in a row, and others are going to be idle 5-10 times in a row, and then randomly that changes. And all of this changes the number of picoseconds that it takes for a clock cycle to complete, which means that each clock cycle is genuinely going to take a different amount of time to complete, and stuff like temperature throttling is completely not at play whatsoever, because we're not talking about chip-wide temperatures, we're literally talking about temperature deltas between transistor a and transistor b.
That makes it a really wonderful source of entropy for cryptographic applications, because the CPU clock is so critical that it's almost never buggy (especially relative to other components that provide entropy), it's also almost impossible to manipulate reliably by an attacker (unless the attacker has an exploit that allows them to set the value of the clock directly - which is possible, but it's a very narrow surface area relative to other entropy sources), and you can completely take advantage of this entropy entirely in userspace, which once again heavily minimizes attack surface area and exposure to bugs.
- strenholme 4 days ago
  
  I’m getting similar findings:
  #include <time.h> #include <stdio.h> #include <stdint.h> int main() { struct timespec foo; int z; uint8_t buffer[512]; for(z=0;z<128;z++) { clock_gettime(CLOCK_REALTIME,&foo); buffer[z * 4] = (foo.tv_nsec >> 24) & 0xff; buffer[z * 4 + 1] = (foo.tv_nsec >> 16) & 0xff; buffer[z * 4 + 2] = (foo.tv_nsec >> 8) & 0xff; buffer[z * 4 + 3] = (foo.tv_nsec) & 0xff; } for(z=0;z<512;z++) { printf("%02x ",buffer[z]); if(z % 16 == 15) {puts("");} } return 0; }
  (code is public domain)
  Here, we see, running it on Windows, at least 1 but of entropy per clock_gettime() call. For people who argue kernel entropy is somehow more secure, perhaps they should become familiar with how kernels before Linux 5.6 or so on some devices had issues where (u)random wouldn’t provide enough entropy to be really secure (people would use haveged to make sure they had enough entropy).