Comment by scosman
1 month ago
This also leads to the unreasonable effectiveness of LLMs. The models are good because they have thousands of years of humans trying to capture every idea as text. Engineering, math, news, literature, and even art/craftmanship. You name it, we wrote it down.
Our image models got good when we started making shared image and text embedding spaces. A picture is worth 1000 words, but 1000 words about millions of images are what allowed us to teach computers to see.
LLMs didn't get good because text is flashy; they got good because text is dense with intention
> effectiveness of LLMs
Is doing dozens of back and forth to explain what we actually want, while the model burns down inordinate amount of processing power at each turn, a model of efficiency or effectiveness ?
It might be convenient and allow for exploration, the cost might be worth it in some cases, but I wouldn't call it "effective".
In many ways LLMs bring the drawbacks of spoken communication back to text.
Regarding effectiveness, LLMs are in a class of their own wrt. their capabilities for general language processing and basic few-shot reasoning.
This also invalidates the "efficiency" question, since the cost of doing those tasks without LLMs is infinity (i.e. you can pay as much as you want, a dolphin is never going to replace the LLM).