Comment by xandrius
5 hours ago
It feels like you probably went too deep in the LLM bandwagon.
An LLM is a statistical next token machine trained on all stuff people wrote/said. It blends texts together in a way that still makes sense (or no sense at all).
Imagine you made a super simple program which would answer yes/no to any questions by generating a random number. It would get things right 50% of the times. You can them fine-tune it to say yes more often to certain keywords and no to others.
Just with a bunch of hardcoded paths you'd probably fool someone thinking that this AI has superhuman predictive capabilities.
This is what it feels it's happening, sure it's not that simple but you can code a base GPT in an afternoon.
If it were not "just a statistical next token machine", how different would it behave?
Can you find an example and test it out?
Wait, you're asking to find and produce a example of a feasible and better alternative to LLMs when they are the current forefront of AI technology?
Anyway, just to play along, if it weren't just a statistical next token machine, the same question would have always the same answer and not be affected by a "temperature" value.
Thats also how humans behave.. I don't see how non determinism tells me anything.
My question was a bit different: if were not just a statistical next token predictor would you expect it to answer hard questions? Or something like that. What's the threshold of questions you want it to answer accurately.
1 reply →
How do non-LLM based World Models behave?
Not sure, can you tell? I feel like you are saying that they may be able to move etc..