Comment by marginalia_nu
9 hours ago
I wonder if a language like Latin would be useful.
It's a significantly much succinct semantic encoding than English while being able to express all the same concepts, since it encodes a lot of glue words into the grammar of the language, and conventionally lets you drop many pronouns.
e.g.
"I would have walked home, but it seemed like it was going to rain" (14 words) -> "Domum ambulavissem, sed pluiturum esse videbatur" (6 words).
I think speculative decoding eliminates a lot of the savings people imagine they're getting from making LLMs use strange languages.
Words <> tokens