Comment by ants_everywhere
4 days ago
You could start with a basic literature review.
https://scholar.google.com/scholar?hl=en&q=llm%20developer%2...
> The only study I’ve seen so far on LLMs and productivity
Why are you guys all citing the same study in this thread? There are several studies including a survey paper.
Personally it's because it's the one that had the best methodology I've seen. Most of the other ones I've seen are kind of a joke. They're mostly self-surveyed, like just asking the participants to say if they were more productive, wrote better code, etc.
Or you've got some that takes like beginner devs that never worked a job and have them run through some unrealistically simple task.
Or a handful that are claiming productivity boosts, but they count things like faster than Google search, or fewer keystrokes.
Or when they measure the "code quality" they just mean the tests passes.
So when that study came out and I read the methodology, I was like, oh, this is interesting. And then the results were also quite surprising, and surprisingly the one with the best methodology had the opposite finding of all the crappy ones I've seen before.
Food for thought.