← Back to context Comment by SoKamil 1 day ago New knowledge cutoff date means this is a new foundation model? 5 comments SoKamil Reply lkbm 1 day ago Yes, but doesn't the token change mean that? clickety_clack 17 hours ago You can train a tokenizer on old data just like you can train a model on old data. wongarsu 15 hours ago But you can't use an old model with a new tokenizer. Changing the tokenizer implies you trained the model from scratch 1 reply → jimmypk 1 day ago [dead]
lkbm 1 day ago Yes, but doesn't the token change mean that? clickety_clack 17 hours ago You can train a tokenizer on old data just like you can train a model on old data. wongarsu 15 hours ago But you can't use an old model with a new tokenizer. Changing the tokenizer implies you trained the model from scratch 1 reply →
clickety_clack 17 hours ago You can train a tokenizer on old data just like you can train a model on old data. wongarsu 15 hours ago But you can't use an old model with a new tokenizer. Changing the tokenizer implies you trained the model from scratch 1 reply →
wongarsu 15 hours ago But you can't use an old model with a new tokenizer. Changing the tokenizer implies you trained the model from scratch 1 reply →
Yes, but doesn't the token change mean that?
You can train a tokenizer on old data just like you can train a model on old data.
But you can't use an old model with a new tokenizer. Changing the tokenizer implies you trained the model from scratch
1 reply →
[dead]