Comment by jp_totsum
13 hours ago
Most people think that using an LLM requires no skill: it is just talking to a machine. I think it is a soft skill that only hours of practice can make you good at. Unfortunately, most of the people I know use LLMs as a search engine: their requests are just short sentences. They get basic results and either they stop there, not impressed or they keep using it as a smart replacement for Google. So they judge what AI can do from their first weak experience. Going further than that would requires more effort: going deep on a subject they know, thinking carefully about the prompt they write, more importantly judging the answer they get and pushing back.
I'm reminded of an interaction I had with a non-technical user. They knew ChatGPT can translate text, and so, to remove spelling mistakes from their (Dutch) text, they would first ask ChatGPT to translate it to English and then ask it to translate it back to Dutch. Mind-blown when I said "you can just say 'fix the spelling and gammar', no need to translate"
This is the level that many non-technical users are operating on.
Right. A life hack carried over from when Google Translate was the best tool, and loop-back through it could be reliably used for longer-form text (similar to how "did you mean?" in Google Search was an excellent spelling corrector for shorter text).
Using LLMs well with human language requires precision.
There are only a few professions that are intrinsically precise with language.
No surprise we're seeing adoption in software programming and mathematics.
If you work with spec files maybe.
For regular work, since we use modern reasoning models, you might as well type your requests with the wrong keyboard layout selected.
It still decodes your intent and completes the requested task. You can work from your phone with voice dictation. It requires no precision at all.
You are essentialy interacting with a SaaS that can be modified and altered at any point by its creator. Treating the LLM as some predictable entity that always works orders of magnitude better with precise language needs to be supported by evidence for each specific case.
Since the vast number of potential users of LLMs are, as you say, poor practitioners of precise language, it is incumbent on LLM providers to continuously iterate on providing LLMs that work better with imprecise language.
Hence, the people who claim that precise language is some 10x - 100x productivty skill seem to ignore the cycle of big LLM providers who are optimizing the models for users who lack those. (One of the early optimizations was the chatbot asking follow-up questions to the user for example).
So I think the people who think that their precise language skills are going to allow them to avoid the jobocalypse are also in a separate bubble of theit own. Precise language skills are going to matter less and less as LLMs increase their hand-holding skills for less equipped users.
This presupposes that the llm will be able to pick up the slack for the users of imprecise language as to me that is a proxy for domain expertise. If so then the human is not needed which does seem like a stretch.
2 replies →
> I think it is a soft skill that only hours of practice can make you good at.
So this, I think, is the crux of it.
When most people, at least in the world of SWE, think of what it means to be skilled at something, they think of knowledge and experience acquired over a much longer timeframe - months to years even. So from that pov, something you can get reasonably good at within just a few hours or days might not seem like much of a "skills moat" at all.
I'm currently seeing a lot of developers and AI industry insiders that do not distinguished between the capabilities of their LLM Agent and the added capabilities of the agent harness. They think the capabilities of the agent harness are within the LLM, and in that confusion they are trying to get their agent to make them an agent harness with all these capabilities. But LLM agent harnesses are brand spanking new, how to make these harnesses are not in the training data of their agent, these developers are more or less vibe begging their agents to make them a harness, so it hallucinates the harness architecture. I am seeing a "agent harnesses" that are delusions, the these developers are not strong and experienced enough to realize they are literally doing the worst work they possibly could... And these idiots are the AI leads.
> think it is a soft skill that only hours of practice can make you good at Yikes