Comment by ekelsen
21 hours ago
I like "between each power of 2, there are the same number of numbers."
So between 1/2 and 1 there are the same number of numbers as between 1024 and 2048. If you have 1024 numbers between each power of 2, then each interval is 1/2048 in the first case and 1 in the second case.
I reality there are usually:
bfloat16: 128 numbers between each power of 2
float16: 1024 numbers between each power of 2
float32: 2*23 numbers (~8 million) between each power of 2
float64: 2*52 numbers (~4.5 quadrillion) between each power of 2