Comment by exporectomy
4 years ago
As long as you're using it to represent what could be physical measurements of real-valued quantities, it's nearly impossible to go wrong. Problems happen when you want stupendous precision or human readability.
Numerically unstable algorithms are a problem too but again, intuitively so if you think of the numbers as physical measurements.
I am regularly reminded of William Kahan's (the godfather of IEEE-754 floating point) admonition: A floating-point calculation should usually carry twice as many bits in intermediate results as the input and output deserve. He makes this observation on the basis of having seen many real world numerical bugs which are corrupt in half of the carried digits.
These bugs are so subtle and so pervasive that its almost always cheaper to throw more hardware at the problem than it is to hire a numerical analyst. Chances are that you aren't clever enough to unit test your way out of them, either.
Yep, floating point numbers are intended for scientific computation on measured values; however many gotchas they hsve when used as intended, there are even MORE if you start using them for numbers that are NOT that. money or any kind of "count" rather than measurement (like, say, a number of bytes).
The trouble is that people end up using them for any non-integer ("real") numbers. It turns out that in modern times scientific calculations with measured values are not necessarily the bulk of calculations in actually written software.
In the 21st century, i don't think there's any good reason for literals like `21.2` to represent IEEE floats instead of a non-integer data representation that works more how people expect for 'exact' numbers (ie, based on decimal instead of binary arithmetic; supporting more significant digits than an IEEE float; so-called "BigDecimal"), at the cost of some performance that you can usually afford.
And yet, in every language I know, even newer ones, a decimal literal represents a float! It's just asking for trouble. IEEE float should be the 'special case' requiring special syntax or instantiation, a literal like `98.3` should get you a BigDecimal!
IEEE floats are a really clever algorithm for a time when memory was much more constrained and scientific computing was a larger portion of the universe of software. But now they ought to be a specialty tool, not the go-to for representing non-integer numbers.
I think you are significantly underestimate the prevalence of floating point calculations, there is a reason why Intel and AMD created all the special simd instructions. Multimedia is a big user for example. You also seriously underestimate the performance cost of using decimal types, we are talking orders of magnitude.
Fair! Good point about multimedia/animation/etc.
There are still a lot of people doing a lot of work in which they hardly ever want a floating point number but end up using it because it's the "obvious" one that happens when you just write `4.2`, and the BigDecimal is cumbersome to use.
I like that idea too. I wonder why Python doesn't use bigdecimals by default. Maybe because it seems to require you to choose a precision?
Notably, this is only true of 64-bit floats. Sticking to 32-bit floats saves memory and sometimes are faster to compute with, but you can absolutely run into precision problems with them. When tracking time, you'll only have millisecond precision for under 5 hours. When representing spacial coordinates, positions on the Earth will only be precise to a handful of meters.
I do a lot of floating point math at work and constantly run into problems either from someone else's misunderstanding, my own misunderstanding, or we just moved to a new microarchitecture and CPU dispatch hits a little different manifesting itself as rounding error to write off (public safety industry).
If you expect bit-for-bit reproducible results, then yea, you'd have to know about the nitty-gritty details. The values should usually still correspond to the same thing in common real world precision though.
> it's nearly impossible to go wrong
It's a matter of time if one doesn't know to look for numerically stable algorithms. Or if one thinks performance merits dropping stability.
https://github.com/RhysU/ar/issues/3 was an old saga in that vein.
Unfortunately, that doesn't work when you have to do:
1 - quantity2 / (quantity1 - quantity2)
... or some such thing. If quantity1 and 2 are similar, ouch!
Not sure if there's a mistake in that expression, since if they're similar, you're already going to get some ridiculously large magnitude (unphysical) result. Maybe you mean calculating the error between two values or convergence testing? In that case, it hardly matters if whether you do
quantity2/quantity1 - 1
or
(quantity2 - quantity1) / quantity1
with double precision and physically reasonable values.
So you have problems if you want a precise answer, you want to display your answer, or if you want to use any of a large number of useful algorithms? That sounds like it’s quite easy to go wrong.
You can't want a precise answer from physical measurements unless you don't know how to measure things. Display should be done with libraries, and numerical instability makes algorithms basically useless, so you pretty much have to be inventing it yourself.