Comment by b112

2 months ago

Right now, all top tier LLMs are as eager, bright 20ish year old interns.

Very gung ho, full of energy, loads of book learning, no real world experience or understanding of why things are as they are.

Leave them to their own devices at your peril. Trust nothing they do.

Yet directly guide them, monitor everything they do, some value emerges.

> Right now, all top tier LLMs are as eager, bright 20ish year old interns. > Yet directly guide them, monitor everything they do, some value emerges.

That's simply not true. I've had some very talented interns, and they are leagues ahead of what the LLMs can do. Not that it matters though, because the point of having interns wasn't to have them produce value. What made the investment worth it was that 12 months down the line I would have a competent colleague that I could have an interesting conversation with. A human person that could challenge some of my blind spots. A person that could take responsibility of something. Maybe not my most important work, but some of it. You don't get ANY of that from the LLM.

I wish people stopped equating LLM to interns or junior engineers. They are tools, and as good they may be at some specific things people do, they also really suck at many more which we wouldn’t find acceptable in humans.

  • Indeed they are tools. But if people/companies treat them as tools that can take a problem a human used to solve, and make them solve it from start to end, then the comparison begins to be necessary.

  • It's merely a way to frame things, in terms of experience and trust. And it highlights how an LLM can code very well, but not truly understand the ramifications of that code.

    • An "eager, bright 20ish year old interns" will grow as you guide them, while LLM will not, they'll only grow when their owner (definitely not us) update them. So, trying to humanizing some tool with human emotion is wrong way to framing it. Treat tool as tool.

I sure hope they stop training them to be spaghetti throwers, please stop and ask questions when there is ambiguity

  • Well let's see. You sell to management. Does management buy:

    1) nuanced tools that talk back, question assumptions, take over decisions, ... oh and expose just how much management knows about the business. Or how little)

    2) a tool that can provide the excuse "we've had our source checked and dealt with the remarks"

    We all know the answer.

In some ways the reality is worse: the same version of an LLM won't get better at its job, even though you might get better at prompting it. Newer versions are trained on more code, which has obvious benefits, and their harnesses are better at taking advantage of tools that were created to keep human coders out of trouble.

The other side of the coin is that coding agents are not maximally productive unless you give them enough rope to potentially hang themselves. Over roughly the past year, the coding agents I use have gone from hot garbage to pretty consistently useful, especially if I find tasks where I can give them a lot of running room. On the other hand, last week I found a case where the coding agent was looping and flailing like it was doing every third try a year ago.

They fail less often, but they fail in the same way.

Have you used a frontier model since 2025? You are underplaying their strength.

  • Kroah-Hartmann has used the closed frontier++ model, and it made up 37 out of 76 vulnerabilities.

    • And the bugs that actually were real rely on a setup so contrived it’s unlikely anyone in the world is impacted.

      It’s still good that some real bugs are being patched but what is being reported to the media is so overblown.

  • We've been seeing "But you're not using the latest model!" over and over again, with every new model supposedly "groundbreaking" and "a game-changer" - just for the general population to conclude a few months later that it once again doesn't live up to the crazy marketing hype.

    Anthropic claimed that Mythos was so good at finding vulnerabilities that it was too dangerous to release to the public. As this post clearly shows: that (only again) simply isn't true. If you believe your favorite flavor of frontier model is the exception, it is up to you to provide proof to back up that claim.

  • Okay so now they're like a top percentile fresh grad on meth. Still a lack of real world experience plus some bizarre failures that illustrate gaping holes in the mental model. Does that description work for you?

    • Well, so basically supercharged fresh grad. CS/math implied.

      In human terms, that's already at least a standard deviation above average person.