Comment by JonChesterfield
7 hours ago
It's written down in the tokenizer. 256 of the tens or hundreds of thousands of tokens correspond to each byte, done.
7 hours ago
It's written down in the tokenizer. 256 of the tens or hundreds of thousands of tokens correspond to each byte, done.