Comment by jansport123
3 hours ago
LLM's are very outcome oriented and i think this is where your observation comes from. You tell LLM you want something, it doesn't even question the premise and just starts calculating 100 different ways to get there. LLM's have knowledge but lack wisdom.
Thanks to improvements in the system prompts/harnesses/RL, the LLMs have gotten way better at questioning the premise and telling you to try something else. It's a huge difference from just 6 months ago.
Nowhere near the level of a suitably-bearded human, but they've gotten pretty good at stopping a lot of bad ideas.
Have you ever asked it to?