Comment by dolebirchwood
3 days ago
I accept there are productivity gains, but it's hard to take "10x" seriously. It's such a tired trope. Is no one humble enough to be a meager 2.5x engineer?
3 days ago
I accept there are productivity gains, but it's hard to take "10x" seriously. It's such a tired trope. Is no one humble enough to be a meager 2.5x engineer?
Even 2.5x is absurd. If they said 1.5x I might believe them.
I'm building an AI agent for Godot, and in paid user testing we found the median speed up time to complete a variety of tasks[0] was 2x. This number was closer to 10x for less experienced engineers
[0] tasks included making games from scratch and resolving bugs we put into template projects. There's no perfect tasks to test on, but this seemed sufficient
Have you evaluated the maintainability of the generated code? Becuause that could of course start to count in the negative direction over time.
Some of the AI generated I've seen has been decent quality, but almost all of it is much more verbose or just greater in quantity than hand written code is/would be. And that's almost always what you don't want for maintenance...
That sounds reasonable to me. AI is best at generating super basic and common code, it will have plenty of training on game templates and simple games.
Obviously you cannot generalize that to all software development though.
5 replies →
One concern is those less experienced engineers might never become experienced if they’re using AI from the start. Not that everyone needs to be good at coding. But I wonder what new grads are like these days. I suspect few people can fight the temptation to make their lives a little easier and skip learning some lessons.
I estimated that i was 1.2x when we only had tab completion models. 1.5x would be too modest. I've done plenty of ~6-8 hour tasks in ~1-2 hours using llms.
Indeed. I just did a 4-6 month refactor + migration project in less than 3 weeks.
I recently used AI to help build the majority of a small project (database-driven website with search and admin capabilities) and I'd confidently say I was able to build it 3 to 5 times faster with AI. For context, I'm an experienced developer and know how to tweak the AI code when it's wonky and the AI can't be coerced into fixing its mistakes.
What's the link?
1 reply →
Downvoted for...confidently saying how successful I was using an AI? I don't get it.
10x probably means “substantial gain”. There is no universal unit of gain.
However if the difference is between doing a project vs not doing is, then the gain is much more than 10x.
I don't know what to tell you, it's just true. I have done what was previously days of BI/SQL dredging and visualizing in 20 minutes. You can be shocked and skeptical but it doesn't make it not true.
There is no x is because LLM performance is non deterministic. You get slop out at varying degrees of quality and so your job shifts from writing to debugging.