Comment by scotty79
8 days ago
> Making an LLM from raw data is value-add. > Distillation is just value extract.
There is a value-add in selecting the valuable parts out of the garbage. And let's face it. Largest models contain a lot of garbage.
8 days ago
> Making an LLM from raw data is value-add. > Distillation is just value extract.
There is a value-add in selecting the valuable parts out of the garbage. And let's face it. Largest models contain a lot of garbage.
I think that's kind of fair, but it still fits within the context of 'some things are value add' and 'more or less than others'.
We ought to identify that and integrate that into our thinking.