Comment by vidarh

They're far more than junior interns. I've had a long languishing project to build an ahead-of-time Ruby compiler. It started as a toy, and a blog series, and I then mostly put it on ice about a decade ago, except for very occasional little rounds of hacking. It self-hosts, but is very limited.

A week or so ago, I gave Claude a task of making it compile rubyspecs. I then asked it to keep making specs pass. I do need to babysit it, but it's doing debugging and work no junior I've ever worked with could be trusted to do. It knows how to work with gdb, and trace x86 assembler. It understands how to read the code of a complex compiler, and modify code generation and parsers that even I - who wrote it in the first place - sometimes find challenging.

It's currently (as I'm writing this) working its way through adding bignum support. Which in Ruby is tricky because it now no longer splits it in two classes - the code need to handle tagged integers that gets auto-promoted to heap allocated objects, that to the user has the same class. I spent the morning swearing at it, but then reset with a clearer description and it produced an extensive plan, and started working through it.

I'll agree it's not great code without a lot of coaxing, but it's doing stuff that even a lot of senior, highly experienced developers would struggle with.

I will agree it needs oversight, and someone experienced guiding it, like a junior developer would, but if I had junior developers producing things this complex, I'd lock them in a basement and never let them go (okay, maybe not).

One of the hardest things, I find, where I will agree it smells of junior developer sometimes, is that it's impatient (once it even said "this is getting tedious") and skipping ahead instead of carefully testing assumptions and building up a solution step by step if you don't tell it very clearly how to work.

I don't think we disagree that much, btw., I just wanted to describe my recent experience with it - it's been amazing to see, and is changing how I work with LLMs, in terms of giving it plenty of scratchpads and focusing on guiding how it works, making it create an ambitious plan to work to, and getting out of its way more, instead of continuously giving it small tasks and obsessing over reviewing intermediate work product.

What I'm seeing often with this approach is that whenever I see something that annoys me scroll past, it's often fixed before I've even had a chance to tell it off for doing something stupid.