Comment by thangalin
3 months ago
Not the OP, but:
https://repo.autonoma.ca/notanexus.git
I don't know the PDF.js library. Writing both the client- and server-side for a PDF annotation editor would have taken 60 hours, maybe more. Instead, a combination Copilot, DeepSeek, Claude, and Gemini yielded a working prototype in under 6 hours:
https://repo.autonoma.ca/notanexus.git/tree/HEAD/src/js
I wrote maybe 3 lines of JavaScript, the rest was all prompted.
> Writing both the client- and server-side for a PDF annotation editor would have taken 60 hours, maybe more.
How do you know? Seems to me you’re making the exact same estimation mistake of the people in the study.
> Instead, a combination Copilot, DeepSeek, Claude, and Gemini yielded a working prototype in under 6 hours
Six hours for a prototype using four LLMs? That is not impressive, it sounds insane and a tremendous mess that will take so long to dig out of the prototype stage it’ll effectively require a rewrite.
And why are you comparing an LLM prototype to a finished product “by hand” (I surely hope you’re not suggesting such a prototype would take sixty hours)? That is disingenuous and skewing the numbers.
> How do you know? Seems to me you’re making the exact same estimation mistake of the people in the study.
I have over 20 years of web development experience and 40 years of general experience writing software. I wrote the authors and they confirmed my thoughts:
"I totally believe it! Per the paper abstract, we find many factors driving results - and one of the factors is how experienced the developers are on the codebase, and how big/complex the codebases are.
"Given that this was a new and unfamiliar domain and new codebase, I would expect there to be much more speedup than the domain we studied!"
> Six hours for a prototype using four LLMs?
They have limits on the number of queries, so I used four different LLMs in tandem to circumvent query limits. I didn't write it four times using four different LLMs.
> it sounds insane and a tremendous mess
I posted the code. It's well organized, has few (if any) encapsulation violations, sticks to OOP quite well, works, and---if I knew the PDF.js API---would be fairly easy to maintain.
Yes, I stand by my claim that writing this annotation editor (PHP, HTML, CSS, and JS) would take me about 60 hours by hand and about 6 hours using the LLMs.