Comment by llm_nerd
7 hours ago
I've literally never heard anyone conflate an agent and a model. Ever.
Often with posts like this I imagine someone had their own confusion and then somehow projected it on everyone else. Like Trump thinking people didn't know about the word groceries or that dumb ends with a b.
I have—frequently—especially among the non-technical crowd.
For example, the recent-ish OpenAI Hugging Face breakout was widely reported as a rogue model escaping. But a model on its own can’t do anything—it’s the agent/harness that escaped. I think it’s an important distinction and I’m glad to see efforts attempting to clear it up.
>But a model on its own can’t do anything—it’s the agent/harness that escaped
An agent/harness "can't do anything" on its own either, so how is saying "an agent escaped" somehow accurate? People talked about the model because it was the model that made the difference. It was specifically the differentiating factor. Not to mention that then you get into the weeds of meaningless arguments like "well actually the agent didn't, TCP packets did"..."well actually it was electrons and photons"...
I knew this would turn into a super boring thing where people will announce that they too misunderstood, therefore everyone does, but this is all very silly nonsense.
> I knew this would turn into a super boring thing where people will announce that they too misunderstood, therefore everyone does, but this is all very silly nonsense.
I don't understand. You said:
> I've literally never heard anyone conflate an agent and a model. Ever.
but when people give you counterexamples suddenly anecdotal experience is boring and silly nonsense?
2 replies →
> I've literally never heard anyone conflate an agent and a model. Ever.
Author is a senior staff engineer. A big part of his job is to help his org's engineers and non-technical decision makers understand basics. My assumption (because it's my experience) is that he is writing this piece because has seen this confusion in others > N times.
And to be fair to those juniors and less-technical folks: big groups of AI phds make the same mistake.
Papers in prominent ML conferences up to like 2024-2025 that were consistently presenting comparisons of proprietary model end-points to open weight models as if they were apples-apples comparisons. Even though there were obvious examples of prop model providers doing "stuff" behind the endpoint as early as Fall 2021.
Most of those papers did not put in the legwork to figure out if/when that "stuff" was happening, or work to figure out what the "stuff" probably was, or conjecture to what extent that "stuff" might overlap with the benchmarks being used, or try to add basic interventions to open-weight models to get a better comparison point, or even simply state the caveat that they don't actually know what happens when they hit a chat completions endpoint!
Not exactly the same thing as conflating agent with model, but 100% conflating "model+software" with "model", which is at the end of the day basically the same confusion. And those papers were being written by and for llm nerds. So if ICML/NeurIPS authors and reviewers aren't careful about "software+model" vs "model" in stating and evaluating their claims, it's unsurprising that some a SWE or non-CS SME also doesn't understand how the LLM software stack works.
It doesn't help that some model providers (OpenAI...) actively obfuscate how the stack works in their marketing.
> I've literally never heard anyone conflate an agent and a model. Ever.
That's because you know enough to disambiguate on the fly, which isn't true for the majority of the population. In other words, you make assumptions about others based on your own condition.
The concern here isn't that someone doesn't know what they're talking about, it's that many of those listening can be misled by the ambiguous wording of people who know very well what they are doing and can even do it deliberately.
> Like Trump thinking people didn't know about the word groceries or that dumb ends with a b.
Again, you're using yourself as source of assumptions about his audience, and worse, you transfer that to a much more complicated subject with unsettled terminology.
> I've literally never heard anyone conflate an agent and a model. Ever.
I recently had to explain it to my brother, who works as a programmer but doesn't read much about technology beyond documentation that solves his immediate problem. (Before anyone comments on whether that attitude is wise: not the point of my comment, and also, this is the reality of how many programmers operate, like it or not).