Comment by ijk
9 hours ago
AI has been turning computer science into biology for the past decade or so. By which I mean that things like neural networks need to be investigated empirically, constructing methodologies and instruments that more closely resemble how fields like biology and medicine have to probe the very complex and messy reality that is beyond our current capacity to fully express in symbolic precision.
Now math gets to deal with that same reckoning. They were already well on their way there with previous Lean proofs, but this has pushed things beyond that horizon and I'm not sure some of the mathematicians are ready for it.
Science has always had an empirical component separate from its theoretical one. For a long time in human history, science was mostly empirical. The periodic table is a great example of mostly empirical observation organized into a pattern. I think the science of the 20th century was the "triumph of theory" so many of us have forgotten what a more empirically driven STEM world is like.
Agreed, I keep thinking to myself that there's a huge mathematical question right in front of us today that is in exactly this vein - all the various nuances of why LLM's work so well is a mathematical question. As far as I know, it's not really understood beyond "we do this basic thing (that makes sense) to predict that a noun is followed by a verb, and then we scale it up a bazillion fold and it can contribute to mathematics research".
As a comparison, classical computing has been scaled up a bazillion fold too, and can do things which are absolutely miraculous, but every layer of abstraction is discretely understandable.
While studying neural networks empirically like biology is one possible approach, there's no reason for that to be the blessed approach other than a combination of inertia and current lack of understanding. It's been only 15 years since AlexNet. Scientists struggled for centuries to model atoms before developing quantum mechanics and later QFT as an accurate quantitative framework.
Also, while biological systems simply exist in nature, artificial neural networks are ultimately mathematical objects with various properties that have yet to be uncovered.
I really like this way of putting it