Comment by swyx

2 days ago

do LoRAs conflict with your distillation?

The architecture is the same so we found that some LoRAs work out-of-the box, but some LoRAs don't. In those cases, I would expect people to re-run their LoRA finetuning with the trainer they've used.