Comment by reissbaker
20 hours ago
At least for RL, you don't need a base model — the rollouts are run in an inference engine with an instruction-tuned model using a chat template. You can start with just that!
20 hours ago
At least for RL, you don't need a base model — the rollouts are run in an inference engine with an instruction-tuned model using a chat template. You can start with just that!
Yeah but they are already RL'd too. No doubt you can further improve an RL'd model by doing your own better RL on top of that. But it's not going to be the same as starting with a base model or instruction tuned model.