Comment by jimbokun
5 days ago
There’s not enough data…which somehow proves your pre-existing bias which goes against the only data we have.
5 days ago
There’s not enough data…which somehow proves your pre-existing bias which goes against the only data we have.
I'm offering my anecdotal evidence as contrary to the loss of productivity piece that has been making the rounds, not as justification that they are wrong, but that we don't have enough data
It's also not the only data, here's one with results in the other direction
https://medium.com/@sahin.samia/can-ai-really-boost-develope...
> Productivity was tracked through metrics such as completed tasks (pull requests), code commits, and successful builds.
Making untested garbage faster to check off tasks quicker. Reopen rate please? New bug task rate? Nobody looked...
> The study also monitored code quality via build success rates. Importantly, increased productivity did not come at the cost of more errors, showing that Copilot helped developers code faster and more accurately.
It builds therefore it works. And the test suite was also AI generated?
Feels like we're back in primary school learning programming.
Classic case of gaming the metrics.
You sound like you haven't given Copilot and friends a thorough chance or evaluation. If you had, you'd know that...
- people don't outsource, they pair-program, because these things cannot do complex tasks on their own
- they are quite good at running tests with coverage, inspecting the results, and fixing both the tests and code
- people make mistakes, expecting AI to be perfect is unreasonable, they are tools, not replacements
> Making untested garbage faster to check off tasks quicker
> Feels like we're back in primary school learning programming.
> Classic case of gaming the metrics.
plain view bias causes others to discount your opinion
1 reply →
Glad to see more data!
It’s interesting they showed the biggest gains for junior developers. The other study showing productivity losses for experienced developers. That suggests these tools are a lot more helpful for junior developers compared to senior developers at the moment.
From my experience, a senior dev who knows how to work these tools will be better than both. We need data on how long, and how much effort into developing the skill, working with AI a developer has put in.
Theore you use it, the more you get a feel for when and when not to use it