Comment by ath3nd
4 days ago
> Those "experienced" actually are just senior code monkeys if u ask me, it's trivial right
Well, it seems you are not open for discussion. There is no reason to disparage the senior devs that participated in the study just because you don't like the results of the study. But the study happened, and it is clear: experienced developers are the ones that suffered from using LLMs.
> but it's just illogical for a junior to get benefits and the seniors don't
Experienced car drivers won't benefit from a youtube tutorial how to drive, junior car drivers might. That's similar to junior developers being potentially the ones who can benefit from the basic things that an LLM can help you with, e.g helping you with syntax and structure your thoughts and write a scaffold to get you started. Those are concerns that experienced developers don't need help with, similarly how experienced drivers don't need youtube tutorials how to shift a gear. There is nothing illogical in that premise. Do you agree?
> i don't think i can do better than the LLM
I most certainly can tell you that there are 1000s of developers that can do infinitely better than any of the current LLMs, and those developers are fairly often senior. It seems like a skill issue you mentioned in the beginning of your post might actually be on your side.
Productivity could just be simple automation. U just describe one part of the whole process. My point stands still. If u cannot get llm to benefit u, u are the problem.
That's like saying if you cannot get a boat to fly, you're a bad pilot.
That study was not conducted well at all. The participants haven’t learned how to use these tools. For example one was interviewed later and said a lot of the time they would wait for an agent and get distracted playing with something irrelevant and then forget to go back until much later. That has solutions they are not aware of to implement.
> For example one was interviewed later and said a lot of the time they would wait for an agent and get distracted playing with something irrelevant and then forget to go back until much later
Counterpoint: the agents are the reason for the distraction.
> That has solutions they are not aware of to implement.
Counterpoint: there are no other current studies that suggest otherwise. Given the impact of LLMs on open source (net negative, maintainers are drowning in slop: e.g curl: https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-s...) maybe it makes sense to be a bit more critical on LLM's supposed gains.
Let's see what we have so far:
- The only study to date suggests a net negative from using LLMs on experienced developers
- OSS maintainers are rejecting AI generated PRs due to low quality
- No other studies have come out so far to suggest otherwise
Based on my anecdotal experience and based on the currently available evidence, for me the conclusion is clear: LLMs and agents are mostly hype.