Comment by thfuran
2 years ago
If the minimal representation of a model of the behavior is "way bigger", why are you disputing that it's more complicated? What's the difference?
2 years ago
If the minimal representation of a model of the behavior is "way bigger", why are you disputing that it's more complicated? What's the difference?
The difference is the capabilities. LLMs don't necessarily need billions of Parameters. In fact, useful models (like the one used in never Apple devices' autocomplete) has only like 50 million. Markov chain... I guess there is probably a reason why we don't use them instead of neuronal networks. Maybe somebody more knowledgeable can enlighten us, but I suspect one might need magnitudes more parameters.
well the neuronal networks are able (when sized appropriately) to memorize the seen markov chains, and more, really.