Comment by torginus
4 months ago
For Fortran, I have the concept of 'immortal code' - code that is generally hard to write, is compatible with everything, and implements an algorithm in a way that's impossible to improve on - or at least doing so would be celebrated as a minor breakthrough.
A lot of numerical optimization code is this - it conforms to a strict 'C' ABI - taking (arrays of) simple floats and ints and outputting the same, so binding it to another higher level language is trivial, so rewriting it makes little sense. If the same algorithm were written in Java, most people would not want to bring in Java as a dependency for their Python/C++/whatever project, but since this is just a tiny C object file, it's happily integrated into everything.
They also tend to be very tricky to get right, I remember reading a paper where the author was adamant that changing the order of a multiply and an add (a mathematically invariant operation) would cause the algorithm to blow up due to the different scales of floating point values involved causing a major loss of precision. I'm sure there's tons of stories like this.
This is the sort of code which took PhD's who studied this exact topic years to get right, even though the actual code often looks unassuming, I would dread the day when I was required to touch it (but I never do - since it always does what it says on the tin)
Swapping addition and multiplication on floating point numbers fails to be invariant.
You are thinking of real numbers, for example as all Cauchy sequences with fractions for coefficients. That is not what a CPU does.