Comment by 999900000999
7 days ago
I'm thinking a customized LLM would write notes in its own hyper compressed language which would allow it to be much much more efficient.
For debugging you could translate it out to English, but if these agents can do stuff without humans in the loop, why do they need to take notes in English?
I can't imagine creating this without hundreds of millions if not billions. I think the future is specialized models
They're literally trained on natural language to output natural language. You would need to create the hyper compressed language first, convert all of your training data to that, and then train the models with that. But token efficiency per word already does vary between different languages, with Chinese being like 30%-40% more efficient than English last I heard
Doesn't this mean the Chinese models have a significant advantage ?
This isn't my domain, but say you had a massive budget, wouldn't a special LLM "thinking" language make sense ?