Comment by chrismorgan
21 hours ago
Most in-browser terminal emulators just use plain old boring DOM, and ignore the issue of perfect columns. Some detect it and wrap offending characters in fixed-width spans. But the entire concept of columns is unsound, doesn’t work at all for a lot of scripts.
It might seem unsound but it’s the concept we’ve built our terminals around.
And there are already definitions and APIs to decide the terminal cell width of different Unicode characters.
I’m not saying it’s easy, because you end up with 3 different string widths:
1. memory length in bytes
2. Number of Unicode characters (which is annoyingly also different to the number of Unicode code points since some codes can be combined)
3. terminal cell width
But as annoying as it is, there are a few relatively mature libraries you can use to solve this problem.
> And there are already definitions and APIs to decide the terminal cell width of different Unicode characters.
And how! I know of four different and mutually-incompatible definitions/behaviours, and I’m confident there will be more; and once you get to things like Indic scripts and Right-to-Left there are no good answers, and no cell-based terminal produces even vaguely acceptable results.
As for meanings of string width, I’d count at least four: Unicode code units, Unicode scalar values, Unicode extended grapheme clusters, and terminal cells.
> And how! I know of four different and mutually-incompatible definitions/behaviours, and I’m confident there will be more;
I’ve not found that to be the case. Usually the issue is more around different libraries being out of date (ie pinned to an earlier version of Unicode)
> and once you get to things like Indic scripts and Right-to-Left there are no good answers, and no cell-based terminal produces even vaguely acceptable results.
That’s because right-to-left isn’t supported and in my opinion shouldn’t be supported my the terminal emulator itself.
Right-to-left support would be handled by the client application, not the terminal emulator. The terminal is just there to offer a grid of Unicode cells. It’s not a writing composition suite.
Just like how the terminal emulator isn’t readline, it isn’t emacs, it isn’t hexdump, and it isn’t vim.
> As for meanings of string width, I’d count at least four: Unicode code units, Unicode scalar values, Unicode extended grapheme clusters, and terminal cells.
I’d already touched on the fact that Unicode widths can vary depending on what you’re measuring. But your Unicode library would/should handle most of the pain there.
When writing software for a terminal, you’re really only interested in the characters you need to loop through, their cell width, and, in some languages, the number of bytes.
As I’ve said previously, I’ve written terminal emulators targeting a variety of different renderers like SDL (so not just js+canvas), as well as complex terminal software like $SHELLs. From what I’ve built over the decades, honestly Unicode support isnt as big problem as it used to be. Sure, there’s plenty of footguns that you need to be aware of, but that’s just the nature of working with technology originally designed in the 60s/70s. Like how you need to be aware of cell widths if you intend to do any manual text alignment in client software. But at least there are libraries to make that task easier.
And if you don’t case about text alignment, then the terminal emulator will handle that for you.
4 replies →