Comment by kmeisthax

As someone who is mildly skeptical of the current wave of LLM hype and thinks it's hitting a wall, I'm noting a lot of parallels between Cyc and ML/DNN systems' founding stories. We even have tales of reward hacking and other weird meta-shenaniganery. One wonders if some renegade at Cycorp ever trained a CycL to English LLM to fake another GOFAI demo with "real" AI.

As for free lunches, the one free lunch the neural network people got was training data. i.e. being able to scrape the entire WWW and build on that[0]. But that's also a free lunch Cycorp wanted, too! Look at all the various attempts at half-open subsets of Cyc. They were half-hearted attempts to solicit training data from randos, sure, but Cycorp still begged for the free lunch anyway.

But I think the big indictment of AI - both old-fashioned and neural network driven - is that they're chasing this nigh-impossible dream of effectively replicating and replacing a person. I still see the same pattern, in both eras, of toy models working well, narrow problem solvers being effective, but then people trying to scale them up to ludicrous sizes by pumping them full of knowledge in an attempt to achieve generality. Even modern LLMs are only slightly more generalist in the sense that they don't immediately fall over and die when a question is phrased in a way that wasn't expected.

The "bitter lesson" of AI is that the only two things that work are search and learning. LLMs and Cyc both attempted to short-circuit this with massive data dumps - the proverbial "knowledge pump" that needed to be primed. Compression is a good learning substitute, so sure, you can "cheat the test" with big data. But, I mean... how many discoveries were made solely by someone mainlining Wikipedia articles like black tar heroin? At some point, you need a problem space to search, and your learning needs to be done online[1] based on the results of that search. Nobody has cracked that code yet.

Maybe the reason why general AI never took off is that there is no such thing as general knowledge. The biggest source of knowledge the AI community has provided to us is knowledge about how to make more AI.

[0] Whether or not this was legal is a different question.

[1] Specifically, while LLMs think analogously to humans (because that's an easily compressible representation) they do not learn analogously to them.