Comment by JonChesterfield
8 hours ago
It's written down in the tokenizer. 256 of the tens or hundreds of thousands of tokens correspond to each byte, done.
8 hours ago
It's written down in the tokenizer. 256 of the tens or hundreds of thousands of tokens correspond to each byte, done.