Comment by dave1010uk
8 hours ago
> Explores what AI cannot
In other words, gradient descent isn't good at combinatorial optimisation. I'm sure the research is better but the hype in the blog post leaves a bad taste.
There must be a version of Rich Sutton’s Bitter Lesson that applies to alternative computing like this, along with all the other exciting specialised hardware we've seen come and go over the years, like expert systems, optical computing, neuromorphic computing, etc.
Something like:
General purpose commodity silicon with rapidly evolving software generally beats specialised hardware.
Software is just so much faster to iterate and improve than hardware. AI is also improving it too (eg AlphaEvolve).
Specialized hardware may give a single, significant improvement that grabs headlines but in the long term, compounding small improvements win.
I don't think they are even referring to gradient descent here. I think they are referring to systems like AlphaEvolve where they use LLMs to give an informed/heuristical guess to try to tackle an otherwise insurmountable search space.
“neuromorphic computer that combines quantum-tunnelling physics with a brain-inspired architecture to find solutions to hard mathematical problems”
I have Bruce Sterling’s Ascendaries: The Best of Bruce Sterling” and… the reality is somewhere here in his stories…
Or take Charles Stross and his Accelerando book.
Do you think that teams behind such projects are avid readers and just fulfill the sci-fi stories? :)
> General purpose commodity silicon with rapidly evolving software generally beats specialised hardware.
All of the Amiga people are sighing right now, as they recall how their beautiful, elegant system synergistically designed with custom chips was outpaced by CPU/memory brute force in the early 90s.
I'm not sure whether these FPGA codes count as specialized hardware.
In hardware Prolog/Kanren/expert systems? That would be possible with libre microcode for Intel, and not this spyware corporate shithole we are living it.
We would be able to switch microcode at boot and set one for security, another one for C performance, others for Lisp performance and so on.
[dead]