Comment by deepsquirrelnet
2 years ago
> This a scratch text area to test the autocomplete in another moment down would be four thousand miles i wish you know please ma am is this new zealand or conversations in another moment down stairs how funny it
^ to me it’s really a stretch that someone could read that and think “markov chains are practically an llm!”
They’re pretty limited, and everyone working on building LLMs knows about them and has probably used them at some point.
They both generate tokens/words on probabilities, but it’s hard to understate how much simpler of a probability model a Markov chain is compared to an llm.
You’ve stated that markov chains have a limited use as a simple chat bot. How do you improve an mc so that it can perform other tasks, like classification, summarization, question answering, complex text parsing, text matching, etc, etc?
The language modeling done in llms has an open pathway to scaling and improving on this statistical model, and it’s incredible to me that this is still opaque to people. I don’t believe that llms are a path to intelligence, but I can certainly recognize that they’re going to emerge as a sophisticated and very useful productivity tool.
> This is where I am right now. Want to go back to my roots. Some people work on old cars, even though they are less efficient. At the same time though, they are more fun to work on than new cars. I've decided to look into Markov chains.
Not sure there’s another side to your argument - we all agree :).
To the author: keep the faith and the excitement if you can, the actual applications of this technology (beyond chatbots) are just getting started.