Comment by codedokode 16 days ago > non UTF-8 encoded strings on input/outputI would just use UTF-8 everywhere. 3 comments codedokode Reply rswail 15 days ago Storing them as 32 bits wide in memory means you can at least index by a codepoint (if not a glyph). codedokode 15 days ago I think you rarely need it. May I know what is your usecase that you need this often? rswail 12 days ago Extract the nth grapheme from a string of Unicode codepoints. Codepoints are 32-bit values.Take into account that some Unicode codepoints work together to combine to form a grapheme which then links to a glyph for display.If you use UTF-8 internally, you will be expanding out to full 32 bits when scanning the vector anyway.So if memory isn't an issue (and most of the time it's not), indexing a vector by codepoint (ie 32 bits) makes more sense from a processing POV.
rswail 15 days ago Storing them as 32 bits wide in memory means you can at least index by a codepoint (if not a glyph). codedokode 15 days ago I think you rarely need it. May I know what is your usecase that you need this often? rswail 12 days ago Extract the nth grapheme from a string of Unicode codepoints. Codepoints are 32-bit values.Take into account that some Unicode codepoints work together to combine to form a grapheme which then links to a glyph for display.If you use UTF-8 internally, you will be expanding out to full 32 bits when scanning the vector anyway.So if memory isn't an issue (and most of the time it's not), indexing a vector by codepoint (ie 32 bits) makes more sense from a processing POV.
codedokode 15 days ago I think you rarely need it. May I know what is your usecase that you need this often? rswail 12 days ago Extract the nth grapheme from a string of Unicode codepoints. Codepoints are 32-bit values.Take into account that some Unicode codepoints work together to combine to form a grapheme which then links to a glyph for display.If you use UTF-8 internally, you will be expanding out to full 32 bits when scanning the vector anyway.So if memory isn't an issue (and most of the time it's not), indexing a vector by codepoint (ie 32 bits) makes more sense from a processing POV.
rswail 12 days ago Extract the nth grapheme from a string of Unicode codepoints. Codepoints are 32-bit values.Take into account that some Unicode codepoints work together to combine to form a grapheme which then links to a glyph for display.If you use UTF-8 internally, you will be expanding out to full 32 bits when scanning the vector anyway.So if memory isn't an issue (and most of the time it's not), indexing a vector by codepoint (ie 32 bits) makes more sense from a processing POV.
Storing them as 32 bits wide in memory means you can at least index by a codepoint (if not a glyph).
I think you rarely need it. May I know what is your usecase that you need this often?
Extract the nth grapheme from a string of Unicode codepoints. Codepoints are 32-bit values.
Take into account that some Unicode codepoints work together to combine to form a grapheme which then links to a glyph for display.
If you use UTF-8 internally, you will be expanding out to full 32 bits when scanning the vector anyway.
So if memory isn't an issue (and most of the time it's not), indexing a vector by codepoint (ie 32 bits) makes more sense from a processing POV.