Comment by bicx
6 days ago
This is exactly what I’ve experienced. For the top-end high-complexity work I’m responsible for, it often takes a lot more effort and research to write a granular, comprehensive product spec for the LLM than it does to just jump in and do it myself.
On the flip side, it has allowed me to accomplish many lower-complexity backlog projects that I just wouldn’t have even attempted before. It expands productivity on the low end.
I’ve also used it many times to take on quality-of-life tasks that just would have been skipped before (like wrapping utility scripts in a helpful, documented command-line tool).
This also accounts for the author of TFA's sense that the smartest people they know are skeptics. Assuming they're being used well, those people spend far more of their time in the high complexity work than they do in the low complexity stuff, so LLMs seem to be more flashy toys than serious tools to them.
That, or higher level in bigger orgs. I’m fairly senior, but I’m also one of two engineers at a startup. Can’t get away from the low-level work.
Bigger org definitely helps, yeah. I'd have loved to have some of these tools available when I was in your shoes!
> On the flip side, it has allowed me to accomplish many lower-complexity backlog projects that I just wouldn’t have even attempted before
This has been my experience at well - AI coding tools are like a very persistent junior-- that loves reading specs and documentation. The problem for AI companies is "automated burndown of your low-complexity backlog items" isn't a moneymaker, even though that's what we have. So they have to sell a dream that may be realized, or may not.
The benchmark project in the article is the perfect candidate for AI: well defined requirements with precise technical terms (RFCs), little room for undefined behavior and tons of reference implementations. This is an atypical project. I am confident AI agent write an HTTP2 server, but it will also repeatedly fail to write sensible tests for human/business processes that a junior would excel at.
I'm currently still somewhat in the AI skeptic camp but you've intrigued me... I'm curious about taking a lesser-known RFC and trying to see what kind of implementation one of the current code-generating models actually comes up with from the spec.