Comment by jobs_throwaway

The nature of LLMs is that you give them input, they give you output. That allows for distillation. They do try to make it harder by hiding the chain of thought stuff, but fundamentally, if your model is publicly available, its distillable.