Comment by thrown_22
4 years ago
Unicode is a mistake that could only have happened in turn of the century America.
It is the distilled essence of the idea that you need to be inclusive of everyone along with a fundamental ignorance of what anyone who isn't American does.
The idea that Chinese characters are glyphs in the same sense of Latin characters can only have come from someone who has never written Chinese.
It is as stupid as demanding a glyph point for each possible integral, e.g. https://quicklatex.com/cache3/8c/ql_9739884527bd893429657272... and https://quicklatex.com/cache3/18/ql_c51509950f58a52253c696a4....
Solutions for English are not solutions for all languages. You can tell because the solutions that were natively invented by people who spoke those languages were _not_ unicode.
JIS, Big5, UHC, and GB all use a codepoint-to-character approach. You're right to point out that many aspects of "multilingual" support are written by people who do not know another language and so end up being hopelessly misguided but it's not really fair to say that Unicode invented this and thrust it upon the CJK world. Every pre-existing system of representing 漢字 had a codepoint table (which Unicode references in their description of each character).
Han Unification was in my view problematic but was driven by technical limitations (then again, if Simplified Chinese characters had also been unified I suspect there would've been more pushback to come up with a better solution, but ultimately Japanese was stuck with being the only one making a major compromise on that front).
I don't think a stroke based or combination system would've been better for many reasons: https://news.ycombinator.com/item?id=32102093. And if you don't trust Americans who at least tried to learn about the subject matter, how much do you trust any other programmer (who has no interest in other languages) to be able to handle a more complicated system for representing and rendering 漢字?
I don't know about Chinese since I'm barely literate in it.
But copying how every Chinese dictionary renders characters as trees of simpler characters seems like a much better approach than the arbitrary letter to number mapping of unicode. That it works well in English is only due to the fact it has no accents.
I can confidently say that native solutions for scripts with accents were _not_ unicode like but overstrike. My grandfather has the source code, in Romanian, of a 1960s computer payment system he worked on which had to deal with both Romanian and Hungarian names.
The combinatorial explosion of possible letters and accents made unicode like encodings an obvious non-starter. Historically names could pick up any accent (some times more than one) on any letter. When you have 26 base letters and 6 possible accents you'd need 26 + 26 * 6 (182) unique representations for single accented letters and 26 + 26 * 6 + 26 ^ 2 (1118) for double accented letters. That makes a language which is fundamentally alphabet based unusable on a keyboard. Something that Unicode is still sweeping under the rug.
> But copying how every Chinese dictionary renders characters as trees of simpler characters seems like a much better approach than the arbitrary letter to number mapping of unicode.
Then why does every natively-developed encoding system in a 漢字-using country not do it that way? For one thing, how would you handle the fact that 食反＝飯 but 食耳＝餌 (and the correct rendering depends on the language and locale of the text being rendered)? How about 辵 (aka 辶, ⻍, ⻌)? There are many issues on top of this one, but this is among the most obvious. In the end you would end up with having your encoding format look like Ideographic Description Sequences (which exist in Unicode) but every rendering library would need to have its own lookup table anyway to produce the correct character. Overlaying accents on top of latin characters (in most European languages) is nowhere near as complicated as combining components to form 漢字.
> I can confidently say that native solutions for scripts with accents were _not_ unicode but overstrike.
Unicode supports combining characters for this reason, though there are separate problems with this approach (some characters look almost identical but semantically should be treated differently -- maybe that is something fonts could deal with, but I suspect "Latin Unification" would've gotten more pushback than Han Unification did). If we want computer systems from different languages and cultures to interoperate there are going to be a few rough edges.
> When you have 26 base letters and 6 possible accents you'd need 26 + 26 * 6 (182) unique representations for single accented letters and 26 + 26 * 6 + 26 ^ 2 (1118) for double accented letters.
No, you don't. Only the most common combinations have their own Unicode number. Most combinations can simply be combined by base and accent ("Mark") numbers. Unicode is not that stupid.
1 reply →
> Every pre-existing system of representing 漢字 had a codepoint table (which Unicode references in their description of each character).
Even typewriters worked with a giant table: https://en.wikipedia.org/wiki/Chinese_typewriter
Han Unification was invented by Chinese people (in Hong Kong iirc), not by Americans. And even before unification, the national standards also had one code point per kanji/hanzi rather than building them up out of radicals, as opposed to the way eg flag emoji are done.
JIS did this in 1978 for instance.
It does appear that computerizing CJK languages has made them very different from handwriting them; native Chinese speakers now constantly forget how to write hanzi. But they did this to themselves.
> It does appear that computerizing CJK languages has made them very different from handwriting them; native Chinese speakers now constantly forget how to write hanzi.
This is separate to the question of encoding -- phonetic-based input systems (IMEs) are a far more likely cause (there are less-widely-used shape-based IMEs which still spit out a Unicode codepoint). The same is happening to Japanese natives, though it should be noted that it's not the case that they cannot write 漢字 normally, they just might forget how to write a relatively rare one (just like how you might forget how to spell a word in English because of a dependence on autocorrect and spellcheck).