← Back to context

Comment by whattheheckheck

14 hours ago

If the Chinese models couldn't Distill from the larger models they'd be at gpt2 or 3 levels

Even if that is true, it doesn't change the reality that they can compete. Also, if we start going that route, American models wouldn't have any quality data to train on if they respected copyright themselves. Their whole product was built on the work of others, on our work, our art.... without compensation, without acknowledgement.

Literally not a single one of these AI companies, regardless of where they are in the world has any right to complain about someone copying their work.

To quote Elon Musk in court:

> OpenAI’s counsel asked Musk whether xAI has ever “distilled” technology from OpenAI.

> Musk: “Generally AI companies distill other AI companies.”

> “Is that a yes?” Savitt asked.

> Musk: “Partly.”

From https://www.interconnects.ai/p/the-distillation-panic which is worth reading in full.