Comment by Fastjur
2 days ago
> To compute the actual speedup – or, rather, slowdown! – provided by AI tools, the researchers compared the developers’ predictions of how long each task would take to the measured completion time.
I'm sorry, but it feels to me like this research has only proven that developers tend to underestimate how long a task is supposed to take, with or without AI.
In no way did they actually measure how much faster a specific task was when performed with and without AI?
What I understand they did is.
You have two tasks:
You ask the dev to estimate both.
Then you randomly tell the dev, ok do Task 1 without AI, and Task 2 with AI.
Then you measure the actual time it took.
Their estimate for AI task missed the mark by 19%, but those without AI were done 20% faster then estimated.
At the time of estimating they didn't know if the task would need to be done with AI or not.