Comment by motoxpro
3 days ago
Ironically, the common (used to be common?) trope about foundation models being picks and shovels and LLM wrappers having no values is probably backwards. ChatGPT is the most valuable AI product and it's just a wrapper around an underlying LLM that is not 10x better than the rest of the models.
I would not define ChatGPT as a wrapper, anything where you are doing actual training/learning and updating weights is by most persons definition, not a wrapper. Just injecting stuff into the context or using RAG is a wrapper because there are no weight updates anywhere.