Comment by blueyes
2 years ago
Cyc is one of those bad ideas that won't die, and which keeps getting rediscovered on HN. Lenat wasted decades of his life on it. Knowledge graphs like Cyc are labor intensive to build and difficult to maintain. They are brittle in the face of change, and useless if they cannot represent the underlying changes of reality.
I wonder to what degree an LLM could now produce frames/slots/values in the knowledge graph. With so much structure already existing in the Cyc knowledge graph, could those frames act as the crystal seed upon which an LLM could crystallize its latent knowledge about the world from the trillions of tokens it was trained upon?
I had the same thought. Does anybody know if there have been attempts either to incorporate Cyc-like graphs into LLM training data or to extend such graphs with LLMs?
From time to time, I read articles on the boundary between neural nets and knowledge graphs like a recent [1]. Sadly, no mention of Cyc.
I'd bet, judging mostly from my failed attempts at playing with OpenCyc around 2009, is that the Cyc has always been too closed and to complex to tinker with. That doesn't play nicely with academic work. When people finish their PhDs and start working for OpenAI, they simply don't have Cyc in their toolbox.
[1] https://www.sciencedirect.com/science/article/pii/S089360802...
1 reply →
There are lattice-based RNNs applied as language models.
In fact, if you have a graph and a path-weighting model (RNN, TDCNN or Transformer), you can use beam search to evaluate paths through graphs.
The problem is not one of KB size. The Cyc KB is huge. The problem is that the underlying inferencing algorithms don't scale whereas the transformer algorithm does.
Lenat was able to produce superhuman performing AI in the early 1980s [1].
[1] https://voidfarer.livejournal.com/623.html
You can label it "bad idea" but you can't bring LLMs back in time.
Why didn't it ever have the impact that LLMs are having now? Or that DeepMind has had? Cyc didn't pass the Turing Test or become superhuman chess and go players. Yet it's had much more time to become successful.
I'm very skeptical of Cyc and other symbolic approaches.
However I think they have a good excuse for 'Why didn't it ever have the impact that LLMs are having now?': lack of data and lack of compute.
And it's the same excuse that neural networks themselves have: back in those days, we just didn't have enough data, and we didn't have enough compute, even if we had the data.
(Of course, we learned in the meantime that neural networks benefit a lot from extra data and extra compute. Whether that can be brought to bear on Cyc-style symbolic approaches is another question.)
2 replies →
Because the goal of Cyc was always to build a genuinely generally intelligent common sense reasoning system and that is a non-sexy long term research project. Being great at chess isn't a measure that's relevant for a project like this, Stockfish is superhuman at chess.
I guess it is funding. Compare funding of Google+Meta to what was/is available to Cyc.
Cyc was able to produce an impact, I keep pointing to MathCraft [1] which, at 2017, did not have a rival in the neural AI.
[1] https://en.wikipedia.org/wiki/Cyc#MathCraft
> Why didn't it ever have the impact that LLMs are having now?
Google has its own Knowledge Graph, with billions of daily views, which is wider but more shallow version of Cyc. It is unclear if LLM user facing impact surpassed that project.
I think before 2022 it was still an open question whether it was a good approach.
Now it's clear that knowledge graphs are far inferior to deep neural nets, but even still few people can explain the _root_ reason why.
I don't think Lenat's bet was a waste. I think it was sensible based on the information at the time.
The decision to research it largely in secret, closed source, I think was a mistake.
I assume the problem with symbolic inference is that from a single inconsistent premise logic can produce any statement possible.
If that is so then symbolic AI does not easily scale because you cannot feed inconsistent information into it. Compare this to how humans and LLMs learn, they both have no problem with inconsistent information. Yet statistically speaking humans can easily produce "useful" information.
Based on the article, it seems like the Cyc had ways to deal with inconsistency. I don't know the details of how they did it, but Paraconsistant Logics [0] provide a general way to prevent any statement from being provable from an inconsistency.
[0] https://en.wikipedia.org/wiki/Paraconsistent_logic
1 reply →
> Compare this to how humans and LLMs learn, they both have no problem with inconsistent information.
I don't have time to fully refute this claim, but it is very problematic.
1. Even a very narrow framing of how neural networks deal with inconsistent training data would perhaps warrant a paper if not a Ph.D. thesis. Maybe this has already been done? Here is the problem statement: given a DNN with a given topology trained with SGD and a given error function, what happens when you present flatly contradictory training examples? What happens when the contradiction doesn't emerge until deeper levels of a network? Can we detect this? How?
2. Do we really _want_ systems that passively tolerate inconsistent information? When I think of an ideal learning agent, I want one that would engage in the learning process and seek to resolve any apparent contradictions. I haven't actively researched this area, but I'm confident that some have, if only because Tom Mitchell at CMU emphasizes different learning paradigms in his well-known ML book. So hopefully enough people reading that think "yeah, the usual training methods for NNs aren't really that interesting ... we can do better."
3. Just because humans 'tolerate' inconsistent information in some cases doesn't mean they do so well, as compared to ideal Bayesian agents.
4. There are "GOFAI" algorithms for probabilistic reasoning that are in many cases better than DNNs.
> Now it's clear that knowledge graphs are far inferior to deep neural nets
No. It depends. In general, two technologies can’t be assessed independently of the application.
Anything other than clear definitions and unambiguous axioms (which happens to be most of the real world) and gofai falls apart. Like it can't even be done. There's a reason it was abandoned in NLP long before the likes of GPT.
There aren't any class of problems deep nets can't handle. Will they always be the most efficient or best performing solution ? No, but it will be possible.
9 replies →
> but even still few people can explain the _root_ reason why.
_The_ (one) root reason? Ok, I’ll bite.
But you need to define your claim. What application?
> _The_ (one) root reason? Ok, I’ll bite.
A "secret" hiding in plain sight.
2 replies →
They are brittle in the face of change, and useless if they cannot represent the underlying changes of reality.
FWIW, KG's don't have to be brittle. Or, at least they don't have to be as brittle as they've historically been. There are approaches (like PROWL[1]) to making graphs probabilistic so that they're asserting subjective beliefs about statements, instead of absolute statements. And then the strength of those beliefs can increase or decrease in response to new evidence (per Bayes Theorem). Probably the biggest problem with this stuff is that it tends to be crazy computationally expensive.
Still, there's always the chance of an algorithmic breakthrough or just hardware improvements bringing some of this stuff into the real of practical.
[1]: https://www.pr-owl.org/
The comment above misses the point in at least four ways. (1) Being aware of history is not the same as endorsing it. (2) Knowledge graphs are useful for many applications. (3) How narrow of a mindset and how much hindsight bias must one have to claim that Lenat wasted decades of his life? (4) Don’t forget to think about this in context about what was happening in the field of AI.
We are living in the future /
I'll tell you how I know /
I read it in the paper /
Fifteen years ago -
(John Prine)
[dead]