Comment by thesz

I think that there is some supporting machinery that uses symbolic computation to guide neural model. That is why chain of thought cannot be restored in full.

Given that LLMs use beam search (at the very least, top-k) and even context-free/context-sensitive grammar compliance (for JSON and SQL, at the very least) it is more than probable.

Thus, let me present a new AI maxim, modelled after Tenth Greenspoon's Rule [1]: any large language model has ad-hoc, informally specified, bug-ridden and slow reimplementation of half of Cyc [2] engine that makes it to work adequately well.

   [1] https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule
   [2] https://en.wikipedia.org/wiki/Cyc

This is even more fitting because Cyc started as a Lisp program, I believe, and most of LLM evaluation is done in C++ dialect called CUDA.