Comment by reliablereason
13 hours ago
Clearly LLMs cant do leaps of intuition since their "intuition" is locked after training ends.
The only way a LLM can come up with new ideas if the "idea" appeared as a generalisation durring training or if it was achieved using reason in chain of thought.
Or some randomness is aomehowntroduced in the output, which happens after every word, unless you set the temperature to zero.
That would not be intuition that is just randomness. Intuition is not randomness.
A jump in intuition comes from automatic processes reorganising the relational structure of conceptual models. There is no reorganisation of the model durring inference.
> There is no reorganisation of the model durring inference.
one could argue that model can reorganize / interact with prior knowledge captured in text form (edit files) hence there can be reorganisation
1 reply →