Comment by thrown_22
4 years ago
I don't know about Chinese since I'm barely literate in it.
But copying how every Chinese dictionary renders characters as trees of simpler characters seems like a much better approach than the arbitrary letter to number mapping of unicode. That it works well in English is only due to the fact it has no accents.
I can confidently say that native solutions for scripts with accents were _not_ unicode like but overstrike. My grandfather has the source code, in Romanian, of a 1960s computer payment system he worked on which had to deal with both Romanian and Hungarian names.
The combinatorial explosion of possible letters and accents made unicode like encodings an obvious non-starter. Historically names could pick up any accent (some times more than one) on any letter. When you have 26 base letters and 6 possible accents you'd need 26 + 26 * 6 (182) unique representations for single accented letters and 26 + 26 * 6 + 26 ^ 2 (1118) for double accented letters. That makes a language which is fundamentally alphabet based unusable on a keyboard. Something that Unicode is still sweeping under the rug.
> But copying how every Chinese dictionary renders characters as trees of simpler characters seems like a much better approach than the arbitrary letter to number mapping of unicode.
Then why does every natively-developed encoding system in a 漢字-using country not do it that way? For one thing, how would you handle the fact that 食反＝飯 but 食耳＝餌 (and the correct rendering depends on the language and locale of the text being rendered)? How about 辵 (aka 辶, ⻍, ⻌)? There are many issues on top of this one, but this is among the most obvious. In the end you would end up with having your encoding format look like Ideographic Description Sequences (which exist in Unicode) but every rendering library would need to have its own lookup table anyway to produce the correct character. Overlaying accents on top of latin characters (in most European languages) is nowhere near as complicated as combining components to form 漢字.
> I can confidently say that native solutions for scripts with accents were _not_ unicode but overstrike.
Unicode supports combining characters for this reason, though there are separate problems with this approach (some characters look almost identical but semantically should be treated differently -- maybe that is something fonts could deal with, but I suspect "Latin Unification" would've gotten more pushback than Han Unification did). If we want computer systems from different languages and cultures to interoperate there are going to be a few rough edges.
> When you have 26 base letters and 6 possible accents you'd need 26 + 26 * 6 (182) unique representations for single accented letters and 26 + 26 * 6 + 26 ^ 2 (1118) for double accented letters.
No, you don't. Only the most common combinations have their own Unicode number. Most combinations can simply be combined by base and accent ("Mark") numbers. Unicode is not that stupid.
>No, you don't. Only the most common combinations have their own Unicode number. Most combinations can simply be combined by base and accent ("Mark") numbers. Unicode is not that stupid.
https://en.wikipedia.org/wiki/List_of_Unicode_characters#Lat...
The most common being literally all of them.
Between Latin-1 Supplement, Latin Extended-A, Latin Extended-B and Latin Extended Additional you have some 700 extra characters of which half are some type of accented letter. I only said you'd need 182 for the six most common European accents. Unicode somehow ends up using 300.
The only people who defend unicode are people who have never looked into the spec.