Comment by kentonv
6 days ago
It took me a few days to build the library with AI.
I estimate it would have taken a few weeks, maybe months to write by hand.
That said, this is a pretty ideal use case: implementing a well-known standard on a well-known platform with a clear API spec.
In my attempts to make changes to the Workers Runtime itself using AI, I've generally not felt like it saved much time. Though, people who don't know the codebase as well as I do have reported it helped them a lot.
I have found AI incredibly useful when I jump into other people's complex codebases, that I'm not familiar with. I now feel like I'm comfortable doing that, since AI can help me find my way around very quickly, whereas previously I generally shied away from jumping in and would instead try to get someone on the team to make whatever change I needed.
> It took me a few days to build the library with AI. ... > I estimate it would have taken a few weeks, maybe months to write by hand.
I don't think this is a fair assessment give the summary of the commit history https://pastebin.com/bG0j2ube shows your work started on 2025-02-27 and started trailing off at 2025-03-20 as others joined in. Minor changes continue to present.
> That said, this is a pretty ideal use case: implementing a well-known standard on a well-known platform with a clear API spec.
Still, this allowed you to complete in a month what may have taken two. That's a remarkable feat considering the time and value of someone of your caliber.
I think the data supports that there were about 5 distinct days when I did a large amount of work on this library, and a sprinkling of minor commits through the rest of the month. Glen's commits, while numerous, were also fairly minor, mostly logistical details around releases.
This library is not the only thing I was working on, nor even the main thing. As the lead engineer of Cloudflare Workers I have quite a few other things demanding my time.
> (...) your work started on 2025-02-27 and started trailing off at 2025-03-20 as others joined in. Minor changes continue to present.
Your analysis is far too superficial to extract anything meaningful. I know for a fact that I have small projects that took me only a couple of days to get done which have a commit history ranging a few months. Also, software is never done. There's always room to refactor, and LLMs turn that into trivial problems. Lastly, is your project still under development if your commits are README updates, linter runs, and renaming variables?
There is a reason why commit history is not used to track productivity.
Is it though?
Would someone of author's caliber even be working on trivial slog item like Oauth2 implementation, if not for the novel development approach he wanted to attempt here ?
For the kind of regular jobs a engineer typically is expected to do, would it give 100% productivity jump ?
Many tools make lesser developers more productive (to a point) but they fail to improve the productivity of talented professionals. Lots of "no/low" code things come to mind. But here's a tool that made kentonv 2x productive at a task that's clearly in his wheelhouse. It seems under the right conditions it can improve the productivity of developers at the opposite end of the spectrum.
What other tools could do that?
1 reply →
The fascinating part is that each person is finding their own way of using these tools from kids to elders and everyone in between no matter what your background or language or whatever is
This. Lots of people talking up agents right now, but the conversational rubber duck thing hits the spot well for me.
Funny thing. I have built something similar recently, that is a 2.1-compliant authorisation server in TypeScript[0]. I did it by hand, with some LLM help on the documentation. I think it took me about two weeks full time, give or take, and there’s still work to do, especially on the testing side of things, so I would agree with your estimate.
I’m going to take a very close look at your code base :)
[0] https://github.com/colibri-hq/colibri/blob/next/packages/oau...
Thanks kentonv. I picked up where you left off, supported with oauth2.1 rfc, and integrated ms oauth to our internal mcp server. Cool to have Claude be business aware
YES!!!! I've actually been thinking about starting a studio specifically geared to turning complex RFPs and protocols into usable tools with AI-assisted coding. I built these using Cursor just to test how for it could go. I think the potential of doing that as a service is huge:
https://github.com/jdbohrman-tech/hermetic-mls https://github.com/jdbohrman-tech/roselite
I think it's funny that Roselite caused a huge meltdown to the Veilid team simply because they have a weird adamancy to no AI assistance. They even called it "plagiarism"
>I have found AI incredibly useful when I jump into other people's complex codebases, that I'm not familiar with. I now feel like I'm comfortable doing that
This makes sense. Are there codebases where you find this doesn't work as well, either from the codebase's min required context size or the code patterns not being in the training data?
In haven’t seen that as a limitation because the agents also are able to grep for keywords to find files to explain things. So they don’t necessarily have to ingest the whole codebase into context.
Matches my experiences well. Making changes to large, complex codebases I know well? Teaching the AI to get up to speed with me takes too much time.
Code I know nothing about? AI is very helpful there
> Though, people who don't know the codebase as well as I do have reported it helped them a lot.
My problem I guess is that maybe this is just Dunning-Kruger esq. When you don't know what you don't know you get the impression it's smart. When you do, you think it's rubbish.
Like when you see a media report on a subject you know about and you see it's inaccurate but then somehow still trust the media on a subject you're a non-expert on.
I think most of this just amounts to the same old good developers vs. bad developers situation that we've been in for decades.
> Like when you see a media report on a subject you know about and you see it's inaccurate but then somehow still trust the media on a subject you're a non-expert on.
Gell-Mann Amnesia https://en.m.wikipedia.org/wiki/Gell-Mann_amnesia_effect
> My problem I guess is that maybe this is just Dunning-Kruger esq. When you don't know what you don't know you get the impression it's smart. When you do, you think it's rubbish.
I see your point. Indeed there are two completely different points of view regarding the output of LLMs:
* Hey, I managed to vibecode my way into a fully working web service with a React SPA after a couple of prompts, and a full automated test suite to boot.
* This project is nowhere as clean as I would have written it, and doesn't even follow my pet coding conventions.
One side lauds LLMs, the other complains they output mainly crap.
The truth of the matter is that the vast majority of software engineers write crap code, as the definition of "crap code" is "something I would have done differently". Opinionated engineers look at the output of LLMs and accuse it of being crap code. Eppur si muove.
> The truth of the matter is that the vast majority of software engineers write crap code, as the definition of "crap code" is "something I would have done differently".
This is certainly a part of it, but I do wonder that even if an LLM “learned” the conventions and preferences of an engineer and spit out “perfectly styled” code, would it be treated as such? I’d wager (a small amount) that it wouldn’t, because part of enjoying the code - for me - is _knowing_ the code. “I wrote it this way because I tried X, then Y, then saw I could do Z, and now I’m familiar with the code in a way that’s more intimate.” Unfamiliar code rarely looks like _really good_, in my opinion.