Comment by andrewstuart
1 day ago
Ok let me rephrase.
Actual tangible real results say different.
I know the anti AI folks would like to believe AI coding results are a mass hallucination but the completed software that I build in record time with deep functionality says otherwise.
I feel LLMs save me time too but I worry my perceptions don't match the reality.
Got some data?
>> Got some data?
There is nothing I could say that would convince a skeptic.
You could repeatedly have developers execute tasks with and without ai (two different tasks. Each subject does the same 2 tasks. Randomize order and what task is done with and what task is done without ai) and show significant differences in duration.
You’d need probably at least 30 to 40 people (there’s ways to estimate this, but this is gut feeling from years ago when I did studies like this)
It would take on a few flaws in tfa which has a low number of people who operate in a codebase that they know very well on issues that they self-selected (and thus already will have a (perhaps subconscious) strategy for).
That would help convincing skeptics.
2 replies →
Or a believer.
How did you measure your before and after?
> Actual tangible real results say different.
Unless you've timed yourself doing the same task once with AI and once without, while having forgotten everything about said task in meantime, you don't have tangible real results that show otherwise.