Comment by symbolicAGI
2 years ago
Doug Lenat, RIP. I worked at Cycorp in Austin from 2000-2006. Taken from us way too soon, Doug none the less had the opportunity to help our country advance military and intelligence community computer science research.
One day, the rapid advancement of AI via LLMs will slow down and attention will again return to logical reasoning and knowledge representation as championed by the Cyc Project, Cycorp, its cyclists and Dr. Doug Lenat.
Why? If NN inference were so fast, we would compile C programs with it instead of using deductive logical inference that is executed efficiently by the compiler.
Exactly. When I hear books such as Paradigms of AI Programming are outdated because of LLMs, I disagree. They are more current than ever, thanks to LLMs!
Neural and symbolic AI will eventually merge. Symbolic models bring much needed efficiency and robustness via regularization.
If you want to learn about symbolic AI, there are a lot of more recent sources than PAIP (you could try the first half of AI: A Modern Approach by Russel and Norvig), and this has been true for a while.
If you read PAIP today, the most likely reason is that you want a master class in Lisp programming and/or want to learn a lot of tricks for getting good performance out of complex programs (which used to be part of AI and is in many ways being outsourced to hardware today).
None of this is to say you shouldn't read PAIP. You absolutely should. It's awesome. But its role is different now.
Some parts of PAIP might be outdated, but it still has really current material on e.g. embedding Prolog in Lisp or building a term-rewriting system. That's relevant for pursuing current neuro-symbolic research, e.g. https://arxiv.org/pdf/2006.08381.pdf.
Other parts like coding an Eliza chatbot are indeed outdated. I have read AIMA and followed a long course that used it, but I didn't really like it. I found it too broad and shallow.
It would be cool if we could find the algorithmic neurological basis for this, the analogy with LLMs being more obvious multi-layer brain circuits, the neurological analogy with symbolic reasoning must exist too.
My hunch is it emerges naturally out of the hierarchical generalization capabilities of multiple layer circuits. But then you need something to coordinate the acquired labels: a tweak on attention perhaps?
Another characteristic is probably some (limited) form of recursion, so the generalized labels emitted at the small end can be fed back in as tokens to be further processed at the big end.
The best thing Cycorp could do now is open source its accumulated database of logical relations so it can be ingested by some monster LLM.
What's the point of all that data collecting dust and accomplishing not much of anything?
It seems the direction of flow would be the opposite: LLMs are a great source of logical data for Cyc-like things. Distill your LLM into logical statements, then run your Cyc algorithms on it.
> It seems the direction of flow would be the opposite: LLMs are a great source of logical data for Cyc-like things. Distill your LLM into logical statements, then run your Cyc algorithms on it.
This is hugely problematic. If you get the premises wrong, many fallacies will follow.
LLMs can play many roles around this area, but their output cannot be trusted with significant verification and validation.
1 reply →
LLM statements (distilled into logical statements) would not be logically sound. That's (one of) the main issues of LLMs. And that would make logical inference on these logical statements impossible with current systems.
That's one of the principal features of Cyc. It's carefully built by humans to be (essentially) logically sound. - so that inference can then be run through the fact base. Making that stuff logically sound made for a very detailed and fussy knowledge base. And that in turn made it difficult to expand or even understand for mere civilians. Cyc is NOT simple.
2 replies →
> The best thing Cycorp could do now is open source its accumulated database of logical relations...
This is unpersuasive without laying out your assumptions and reasoning.
Counter points:
(a) It would be unethical for such a knowledge base to be put out in the open without considerable guardrails and appropriate licensing. The details matter.
(b) Cycorp gets some funding from the U.S. Government; this changes both the set of options available and the calculus of weighing them.
(c) Not all nations have equivalent values. Unless one is a moral relativist, these differences should not be deemed equivalent nor irrelevant. As such, despite the flaws of U.S. values and some horrific decision-making throughout history, there are known worse actors and states. Such parties would make worse use of an extensive human-curated knowledge base.
An older version of the database is already available for download, but that's not the approach you want for common sense anyway, no one needs to remember that a "dog is not a cat".
1 reply →
OpenCyc is already a thing and there's been very little interest in it. These days we also have general-purpose semantic KB's like Wikidata, that are available for free and go way beyond what Cyc or OpenCyc was trying to do.
I think military will take over his work.Snowden documents reveled the cyc was been used to come up with Terror attack scenarios.
> If NN inference were so fast, we would compile C programs with it instead of using deductive logical inference that is executed efficiently by the compiler.
This is the definition of a strawman. Who is claiming that NN inference is always the fastest way to run computation?
Instead of trying to bring down another technology (neural networks), how about you focus on making symbolic methods usable to solve real-world problems; e.g. how can I build a robust email spam detection system with symbolic methods?
> Instead of trying to bring down another technology (neural networks), how about you focus on making symbolic methods usable to solve real-world problems; e.g. how can I build a robust email spam detection system with symbolic methods?
I have two concerns. First, just after pointing out a logical fallacy from someone else, you added a fallacy: the either-or fallacy. (One can criticize a technology and do other things too.)
Second, you selected an example that illustrates a known and predictable weakness of symbolic systems. Still, there are plenty of real-world problems that symbolic systems address well. So your comment cherry-picks.
It appears as if you are trying to land a counter punch here. I'm weary of this kind of conversational pattern. Many of us know that tends to escalate. I don't want HN to go that direction. We all have varying experience and points of view to contribute. Let's try to be charitable, clear, and logical.
I am desperately vetting your comment for something I can criticize. An inadvertent, irrelevant, imagined infraction. Anything! But you have left me no opening.
Well done, sir, well done.
2 replies →
>> If NN inference were so fast, we would compile C programs with it instead of using deductive logical inference that is executed efficiently by the compiler.
> This is the definition of a strawman.
(Actually, it is an example of a strawman.) Anyhow, rather than a strawman, I'd rather us get right into the fundamentals.
1. Feed-forward NN computation ('inference', which is an unfortunate word choice IMO) can provably provide universal function approximation under known conditions. And it can do so efficiently as well, with a lot of recent research getting into both the how and why. One "pays the cost" up-front with training in order to get fast prediction-time performance. The tradeoff is often worth it.
2. Function approximation is not as powerful as Turing completeness. FF NNs are not Turing complete.
3. Deductive chaining is a well-studied, well understood area of algorithms.
4. But... modeling of computational architectures (including processors, caches, busses, and RAM) with sufficient detail to optimize compilation is a hard problem. I wouldn't be surprised if this stretches these algorithms to the limit in terms of what developers will tolerate in terms of compile times. This is a strong incentive, so I'd expect there is at least some research that pushes outside the usual contours here.
The point is that symbolic computation as performed by Cycorp was held back by the need to train the Knowledge Base by hand in a supervised manner. NNs and LLMs in particular became ascendant when unsupervised training was employed at scale.
Perhaps LLMs can automate in large part the manual operations of building a future symbolic knowledge base organized by a universal upper ontology. Considering the amazing emergent features of sufficiently-large LLMs, what could emerge from a sufficiently large, reflective symbolic knowledge base?
That what I have settled on. The need for a symbolic library of standard hardware circuits.
I’m making a sloppy version that will contain all the symbols needed to run a multi-unit building.