Only know the discussion superficially, but from what I understood, the problem was as follows: Chinese Hanzi, Japanese Kanji and Korean Hanja scripts have the same historical roots (Han) and still look very similar when you write them — however, they are fully separate scripts with different use of letters and optical differences that are important.
However, when encoding them into Unicode, the consortium suddenly discovered the importance of being frugal with character IDs and only defined a single "Han" sequence of characters for all three scripts. You need to select a matching font to actually display the text in the script that's intended.
This makes using those characters an unnecessary hassle and also came over as a cultural faux-pas to some (essentially like encoding Greek Eta, Latin H and Cyrillic En as the same character, because they look similar and the scripts have a common history)
<s>It also looks a bit hypocritical when Unicode otherwise goes the opposite way and has the space to encode dozens of variants of the Latin alphabet or — as we see here — several categories of dashes.</s> (Edit: Han unification was done when Unicode was still 16 bit, so there were real space constraints — so no hypocrisy actually)
I honestly agree with the arguments they made to defend it, but then they tied themselves in loops trying to defend things like separate Greek and Cyrillic scripts and making superfine distinctions about how that isn't the same phenomenon. But they couldn't merge those scripts together now even if they wanted to, because of the stability policy. And the existence of the blackletter, small-caps, superscript, etc. characters seems entirely indefensible to me. All font rendering issues; it's clearly the same symbol with the same fundamental meaning.
Really, they needed to come up with the idea of variation selectors way earlier, and use them aggressively. And design something more consistent for character composition, and skip the precomposed characters (those really are glyphs representing a cluster; being able to treat the cluster as an atomic entity versus its components doesn't add actual semantic information to the text).
If I'm writing a text about Russian language in English, I can easily include an example in Russian: "привет, как дела?" and it will be displayed correctly.
But, if I was writing about Mandarin Chinese in Japanese, I just wouldn't be able to do it using a simple text input (like HN comment form). I would need more advanced formatting tools to make sure that both Chinese and Japanese characters are displayed correctly.
Because, as far a I understand, one Unicode symbol could correspond to both a Hanzi character and a Kanji character. Which character will be displayed is only determined by font. Since HN doesn't allow me to select different fonts for different parts of my comment, it is simply impossible to create a comment that includes both Japanese and Chinese characters.
Only know the discussion superficially, but from what I understood, the problem was as follows: Chinese Hanzi, Japanese Kanji and Korean Hanja scripts have the same historical roots (Han) and still look very similar when you write them — however, they are fully separate scripts with different use of letters and optical differences that are important.
However, when encoding them into Unicode, the consortium suddenly discovered the importance of being frugal with character IDs and only defined a single "Han" sequence of characters for all three scripts. You need to select a matching font to actually display the text in the script that's intended.
This makes using those characters an unnecessary hassle and also came over as a cultural faux-pas to some (essentially like encoding Greek Eta, Latin H and Cyrillic En as the same character, because they look similar and the scripts have a common history)
<s>It also looks a bit hypocritical when Unicode otherwise goes the opposite way and has the space to encode dozens of variants of the Latin alphabet or — as we see here — several categories of dashes.</s> (Edit: Han unification was done when Unicode was still 16 bit, so there were real space constraints — so no hypocrisy actually)
See here: https://en.wikipedia.org/wiki/Han_unification
I honestly agree with the arguments they made to defend it, but then they tied themselves in loops trying to defend things like separate Greek and Cyrillic scripts and making superfine distinctions about how that isn't the same phenomenon. But they couldn't merge those scripts together now even if they wanted to, because of the stability policy. And the existence of the blackletter, small-caps, superscript, etc. characters seems entirely indefensible to me. All font rendering issues; it's clearly the same symbol with the same fundamental meaning.
Really, they needed to come up with the idea of variation selectors way earlier, and use them aggressively. And design something more consistent for character composition, and skip the precomposed characters (those really are glyphs representing a cluster; being able to treat the cluster as an atomic entity versus its components doesn't add actual semantic information to the text).
If I'm writing a text about Russian language in English, I can easily include an example in Russian: "привет, как дела?" and it will be displayed correctly.
But, if I was writing about Mandarin Chinese in Japanese, I just wouldn't be able to do it using a simple text input (like HN comment form). I would need more advanced formatting tools to make sure that both Chinese and Japanese characters are displayed correctly.
Because, as far a I understand, one Unicode symbol could correspond to both a Hanzi character and a Kanji character. Which character will be displayed is only determined by font. Since HN doesn't allow me to select different fonts for different parts of my comment, it is simply impossible to create a comment that includes both Japanese and Chinese characters.