Comment by Balinares
8 days ago
Overheard recently: "Thanks to AI we're producing more code and more MRs, faster than ever, but the milestones aren't getting hit any sooner. Actually the opposite, if anything."
I wonder how widespread that phenomenon is. Perhaps it's no wonder the prominent actors are trying to rush to IPO...
Goodharts law. The metrics were always measuring the wrong thing, and now that we've finally optimized for the wrong thing successfully management will be forced to admit it and move on to another, slightly different, metric that doesn't actually equate to shareholder value.
It doesn't matter what the line actually measures, just that it goes up.
That's exactly what's happening. Many claim they are more productive with AI, but individual rise in productivity just doesn't translate to projects being completed any sooner.
And by "projects", I mean corporate ones with big teams involved. Hobby projects actually do get finished much faster.
In the corporate world, writing code was always a small chunk anyway. Iirc something like 30% of employee time. Getting 50% faster there still only gains back 10% of your time.
To further complicate matters, you had to spend on AI software and potentially additional on legal/risk/security/compliance to enable that.
The smaller the company, the bigger that % of coding is of total time (all the way down to hobby where the majority of time is spent coding)
Where I work, I would guess that, out of the amount of person-hours spent on a project, less than 5% of it is the cumulative time spent designing and writing code. There's so much other stuff going on: Requirements gathering, "aligning" with other teams, human reviews, QA, presenting to executives, waiting for approval from executives, reporting status, deploying to staging, internal dogfooding, slow ramp-ups to production... This is how projects that are 50 lines of code take 3 months to deploy. AI is helping reduce the time spent on that 2%.
We're planning for Q1 and don't have requirements yet for features, but are expected to have detailed estimates ready... This has happened before AI, this is happening after AI. The problem has rarely been "it takes too long to build ". The problem was "what do you want me to build? wait, no, not like that. now we must iterate"
My take from working at a Big Corp is that individuals using coding agents can increase velocity substantially and produce good quality work, assuming they are proficient with the tools. But it falls apart quickly when you have a team trying to work together.
imo we either need to centralize the agent and and submit plan, spec, reference doc MRs rather than submitting code changes. Or develop SCM systems/workflows that incorporate plan/spec/reference/prompt metadata with code so intent can be factored into merges.
Was that at Microsoft by any chance?