Comment by thangalin

> How do you know? Seems to me you’re making the exact same estimation mistake of the people in the study.

I have over 20 years of web development experience and 40 years of general experience writing software. I wrote the authors and they confirmed my thoughts:

"I totally believe it! Per the paper abstract, we find many factors driving results - and one of the factors is how experienced the developers are on the codebase, and how big/complex the codebases are.

"Given that this was a new and unfamiliar domain and new codebase, I would expect there to be much more speedup than the domain we studied!"

> Six hours for a prototype using four LLMs?

They have limits on the number of queries, so I used four different LLMs in tandem to circumvent query limits. I didn't write it four times using four different LLMs.

> it sounds insane and a tremendous mess

I posted the code. It's well organized, has few (if any) encapsulation violations, sticks to OOP quite well, works, and---if I knew the PDF.js API---would be fairly easy to maintain.

Yes, I stand by my claim that writing this annotation editor (PHP, HTML, CSS, and JS) would take me about 60 hours by hand and about 6 hours using the LLMs.