Comment by mmasu
7 days ago
While I don’t agree with you, I keep a healthily skeptical outlook and am trying to understand this too - what is the hard data? I saw a study a while ago about drops in productivity when devs of OSS repos were AI assisted, but sample size was far too low and repos were quite large. Are you referring to other studies or data supporting this? Thanks!
I, individually, am certainly much more productive in my side projects when using AI assistance (mostly Claude and ChatGPT). I attribute this to two main factors:
First, and most important, I have actually started a number of projects that have only lived in my head historically. Instead of getting weighed down in “ugh I don’t want to write a PDF parser to ingest that data” or whatever, my attitude has become “well, why not see if an AI assistant can do this?” Getting that sort of initial momentum for a project is huge.
Secondly, AI assistants have helped me stretch outside of my comfort zone. I don’t know SwiftUI, but it’s easy enough to ask an AI assistant to put things together and see what happens.
Both these cases refer almost necessarily to domains I’m not an expert in. And I think that’s a bigger factor in side projects than in day jobs, since in your day job, it’s more expected that you are working in an area of expertise.
Perhaps an exception is when your day job is at a startup, where everyone ends up getting stretched into domains they aren’t experts in.
Anyways, my story is, of course, just another anecdote. But I do think the step function of “would never have started without AI assistance” is a really important part of the equation.
Ithink there are also 2 factors to this.
1. Learning curve: Just like any skill there is a learning curve on how to get high quality output from an LLM.
2. The change in capabilities since recent papers were authored. I started intensively using the agentic coding tools in May. I had dabbled with them before that, but the Claude 3.7 release really changed the value proposition. Since May with the various Claude 4, 4.1 and 4.5 (and GPT-5) the utility of the agentic tools has exploded. You basically have to discard any utility measure before that inflection point, it just isn't super informative.