Comment by ekelsen

1 day ago

I like "between each power of 2, there are the same number of numbers."

So between 1/2 and 1 there are the same number of numbers as between 1024 and 2048. If you have 1024 numbers between each power of 2, then each interval is 1/2048 in the first case and 1 in the second case.

I reality there are usually:

bfloat16: 128 numbers between each power of 2

float16: 1024 numbers between each power of 2

float32: 2*23 numbers (~8 million) between each power of 2

float64: 2*52 numbers (~4.5 quadrillion) between each power of 2