Comment by wongarsu
17 hours ago
But you can't use an old model with a new tokenizer. Changing the tokenizer implies you trained the model from scratch
17 hours ago
But you can't use an old model with a new tokenizer. Changing the tokenizer implies you trained the model from scratch
A little bit of post-training will fix that. Folks on /r/LocalLLaMa have been making effective finetunes with diff. tokenizers for years.