Comment by sitkack
5 months ago
The default system prompts are tuned for the naive case. LLMs being all purpose text handling tools, can be reprogrammed for any behavior you wish. This is the crux of skilled use of LLMs.
The better the LLMs get, the worse the average prompt quality.
Yep. It's fairly trivial to prompt an LLM to say "I don't know" when it doesn't know something.