Comment by ventana
10 hours ago
It's more like ∞x (or N/Ax if you prefer) because the majority of the projects I did with LLM agents wouldn't have existed without them, because I would've never found enough time to work on them.
One of the latest things I made with Claude was a tool that allowed me to move a bunch of very low traffic Cloud Run services to a single VPS without losing any of the Cloud Run benefits such as easy Docker-based deployment and automatic certificate provisioning. I thought about making something like that for quite some time, and Claude finally made it possible, which makes me quite happy.
The fun thing here is that no other soul genuinely cares about it, or any other code I might publish. The code, especially AI generated, is so cheap that if anyone wants to repeat my steps to get rid of Cloud Run services, they will probably vibe-code their own tool instead of figuring out how to use mine, just like I did that instead of spending time on learning Dokku or similar solutions.
So, yes, 10x and more, but no one cares about the result, which makes the whole 10x measurement less useful.
There have been plenty of libraries/tools that got people from "I would like to do foo" to "oh, this tool makes foo possible in under 10 hours, I should start working on foo" in the past. LLMs have done this for many foos, which is great.
But I'm with hansvm - I haven't actually seen anyone plausibly maintain 10x. 10x is different from getting people past their activation cost.
The incredulity at 10x claims is often unearned because how much do these skeptics actually notice and appreciate the depth of work of ten developers collaborating on something (if not their own org)? Dev output slips by quietly. There are reams of unnoticed projects even at the scale of a life’s work.
This doesn't pass a sniff test at any small organization. And wouldn't these devs see this 10x claim themselves?
My small organization is noticing output increasing. We're excited about it. I’m not sure about 10x… Like others have mentioned, it’s difficult because you have to measure different workloads.
I build things I never would have. My tooling is better and more robust than ever. I verify and test my work better than ever. I fix more bugs than I used to simply because no one needs to care if it fits into a cycle. I explore and solve more problems in more parts of the application, even if I don’t write code. I take better care of our infrastructure. Performance goes up, bugs go down, AWS resources scale back, costs go down. I’ve paid for my AI usage in scaled back resources several times over at this point.
It might not be 10x but it’s a significant multiple.
I'm assuming the devs are seeing 10x code generation and equating that to the improvement.
It's when they practically ignore the rabbit holes where it's suspect. I'm definitely seeing speed ups. I troubleshot a linux system yesterday with minimal effort using a local llm. It likely would have taken me a few hours to locate all the docs & testing procedures. the llm did it with only a few prompts. To ensure it did it correctly, I had to interrogate it a few times before letting it proceed.
Humans make really bad scientists, and it takes a lot of effort to properly catalog and provide statistics for these things.
There is an improvement, but I doubt any random dev can give a real estimate since before LLMs they couldnt really give you a real estimate anyway. I do know when I encounter a bug now, debugging is almost immediately possible.