Comment by vatsachak
1 day ago
LLMs sure, but AlphaZero had no visual cortex yet can smash Magnus Carlsen easily.
I think that we're not that far away from AI that can be superhuman at all facets of theorem proving.
I think that we're far away from an AI that can create good abstractions and construct a theory to prove theorems.
A convolutional neural network is really somewhat like a visual cortex. Obviously AlphaZero doesn't literally have a visual cortex -- actual literal visual cortices are features of actual literal brains made out of meat -- but it definitely has something that does something akin to visual processing, in a way that LLMs don't. Or at least they don't on the face of it; maybe well trained large enough LLMs have effectively implemented something kinda-visual-cortex-like on top of the transformer architecture.
(I bet there are people at all the big AI labs working on ways to incorporate something more CNN-like into LLMs somehow.)
I believe the architecture for convolution neural networks were directly inspired by how vision works and some of the core design choices map onto real features of the visual cortex.
That's fair. Two things that are heavily underrated are architecture and encoders
> LLMs sure, but AlphaZero had no visual cortex yet can smash Magnus Carlsen easily.
Chess does not require a visual cortex to play. People have been playing by mail with algebraic notation for centuries.
This feels like a non sequitur
Everyone, including blind people use their visual cortex to play chess
AI only works in the parts of reality that have been defined into granular atomic units. Which is by definition an approximation of reality.
AI so far has almost no way to interact with non definitional non quantitized reality. So novel space is still deeply out of its domain.
Recombination of known spaces it will probably continue to make pure war dial breakthroughs in though.
I wish we’d tackle a post Mathematics world where we’d account for number theory not being accurate abstraction of reality (I.e. there is no 123 only 1ish 2ish 3ish with many sub properties of any given unit we are ignoring)
There's actually a lot of math trying to describe the types of space you mention - non quantized, non 'granular', they're not made of points, or distances (metrics). One deep idea is to define space through which symmetries hold (Klein's Erlangen program). Topology itself is not interested in distances per se, only properties of a space invariant under homeomorphisms (a fancy way of saying you can continuously deform a cup to be a donut). Thurston's Geometrization Theorem outlines the 8 geometries that a closed 3 manifold can have. Topos theory studies space in a very general setting that connects logic to it. You may like the books "The Shape of Space" and "Surreal Numbers". The Numbers 1, 2, 3 is not the only in which mathematicians abstract reality.
I know I failed to explain this correctly and was downvoted for it
But I think you got my point. “Granularism” itself is an approximation of a specific set of dimensions of space.
Tokenizing reality so far is somewhat incompatible with say real time forces (new dimensionalism as you sort of describe here) Even if there are granularities representations of them.
So whatever hasn’t been granulized so far AI can’t understand because it would need to verify in reality through observation if the new units of granularity align with outcomes.
This is of course possible to some degree but runs into a paradox if it is a portion of reality than is irreducible or works in a fundamentally “non granular” way.
I have no proof of this but of course already at the quantum level we are running into to “non granular” realities
2 replies →