Comment by YeGoblynQueenne

>> Crude attempts, such as OpenCyc and other formal ontological reasoner systems, would need trillions of low level rules to have a rough approximation of the world model as complex as that of a human child. AI with trillions of parameters could probably start getting to the point where there's parity with human scale, but even if you turned the entire planet earth into computronium and turned it toward the task of understanding all the theory and science of the universe, there will always be far more left to explore and understand than the sum total of all knowledge.

I thought CyC was a heroic attempt doomed to failure but not because it was rule-based. IIUC it used (custom) first-order logic and that's as expressive as expressive can be. There's no reason why a sufficiently large first-order rule-base could not capture every human concept. There's also no reason why "trillions of parameters" would be any better at that than "trillions of rules". It really comes down to what those rules or parameters are encoding.

What doomed CyC to failure, I think, is that its rule-base was mainly manually encoded. CyC was basically the world's biggest ever expert system, and it came with the biggest ever knowledge acquisition bottleneck. I don't think there's any magick to my claim, either. Human minds can't handle the complexity of a few dozen, let alone a few million, interconnected rules without making mistakes. It's hopeless trying to create such a system by hand. From a certain point onward you have no idea what your system can and can't do, because you have no idea what information it has and hasn't access to.

But it's hopeless trying to create such a system by chance, too, i.e. by feeding the system all the data we can find in the hope that it will somehow spontaneously acquire all the knowledge we want it to; much of which is not even in the deductive closure of the information we feed it (and LLMs are not deductive inference engines, unlike expert systems).

Some kind of automatic knowledge acquisition is clearly a much better idea than manually coding rules by hand, but I don't see how peta-scale machine learning has moved the needle much either. We're still stuck with systems that can do some spectacular things but can't do simple things. Or things that look simple to us.