Comment by ACCount39
5 hours ago
All LLMs understand natural language. All LLMs understand examples. That's honestly more compatibility than you get nearly anywhere, in anything.
The reason why advanced prompting is a moving target is that a lot of prompting is "use extra instructions to compensate for specific ways in which the target LLM is weak or prone to errors". And guess what? LLMs get better over time - obsoleting your advanced prompting.
"Tune a prompt to death for the specific task and the specific model" gets you better performance in the moment, but "trust LLM to be smart" ages a lot more gracefully.
"but "trust LLM to be smart" ages a lot more gracefully."
That it doesn't even work now.
The word 'smart' there is actually doing a lot of heavy lifting, it's entirely contextualized.
So much AI discourse tacitly assumes that there's an objective quality that corresponds to "being smart", rather than a chaotic patchwork of extremely contextual social practices and expectations.
Yes, this is it exactly.
I totally understand what a developer might mean by 'smart' but we should have the self awareness to recognize it's barely meaningful outside of what we do.
It's nowhere near that simple. For instance, models used to be WAY better at writing, until the labs decided that coding ability was a better thing to focus on, and trained successor models accordingly.