Comment by athrowaway3z
14 hours ago
I agree with the disconnect between the professionals, but i have yet to see anything that makes me believe that there is a class of 5000 internal people who are that much further than the top users are.
The internal models are more eager, and there is the scale difference of being able to spin up 10.000 vs 10, but to get something valuable out requires skill.
Agents that produce good results 90% of the time vs 93% of the time are not enough of a difference. Multiplied too many times, that error rate is too high for scale to become "magic".
Finding proofs is one of the exceptions.
Just look at the codex/claude harness. They regularly lag behind 'external' users.
I feel like you're taking it somewhat personally as a statement on ability or intelligence levels. It's really just about the scale difference and the access to that infrastructure. That is what enables that class to look at and think about problems differently.
Having 10,000 (or more) agents working for months on a problem is categorically different from the type of usage most people are familiar with. The complexity explodes to a point that requires the "archaeology" he mentions for people to even try to understand what's going on.
This is the key insight. Those capabilities are currently limited to frontier labs and literally cost them millions of dollars to achieve. But as we well know, computing costs and algorithms are optimized rapidly and soon those capabilities will be more widespread. We are at an inflection point where we can design systems that are able to go and do things that we don't fully understand, and they will handily outpace our forensic bandwidth.
> But as we well know, computing costs and algorithms are optimized rapidly and soon those capabilities will be more widespread. We are at an inflection point
It seems we’re perpetually in this state.
As far as I can tell, we seem to keep moving past states and redefining them while we do.
2 replies →
I hear you, but also, "a broken clock is right twice a day." And really, there are people still alive who have more or less witnessed the entire rise of the machines.
He’s just trying to pitch himself as part of the priest class.
My understanding is he is not talking about "top users" as being further ahead in skill, he is talking about access.
The kind of FOMO he is triggering is the same kind that wanting to be in an exclusive country club gives a certain kind of businessman. One simply can't reproduce that network outside of the exclusive access, no matter how skilled they are.
Lauren Tan from SpaceXAI/Cursor/Grok has been on a few podcasts recently. I was surprised how much info she shared, and she sounded quite honest about what works and what doesn’t, in the context of building grokbot. I wouldn’t be able to do all the verifications she does, with my token budget. But it doesn’t seem like rocket science either (pun not intended).
I agree, and to keep sane I enjoy reading Boris Cherny discuss how he made his own discoveries while designing and building Claude Code cli.
To me he seems half led by measurements (like output quality) and half by his own learning-as-he-goes (like his inventing "plan mode" after an evening of vibe coding and realizing it kept implementing too early in the process).
90% vs 93% is a big deal IMO, especially when you're trying to feed outputs into inputs.
I think the point being made is that when you're feeding outputs into inputs, both 93% and 90% rapidly collapse to 0% without some way to recover.
You might be thinking along the lines of the O-ring theory. Indeed long supply chains accumulate errors exponentially.
However the origins of theory explored the outcome of disastrous and irreparable outcome of space shuttle exploding, lives lost, and billion of dollars destroyed.
When LLM goes off the rails the cost is much lower, and retry is on the table. In fact our core competency shifted from writing code to creating environment where agents can retry cheaply and course correct based on feedback.
A modest increase in the number of turns it takes to reach decorehenve could make a difference between Decoherence and reaching the a grounded checkpoint.
1 reply →
Which is precisely what I'm responding about above. 93% actually collapses much slower than 90% since the rate of reduction is based on that of all of the previous rounds, not just the current chance. E.g., this is the difference between 14 rounds before an expected failure instead of 10. An extra 4 rounds deep at the same error rate is a big step forward despite a small change in the base percentages. Add just few more percentage points again and you've more than doubled the depth.
I.e. it's precisely when you look at these chained that the difference is actually very large. It's really only the 1st turn or two of prompts where the difference seems negligable.
4 replies →
First off, the 90% to 93% success rates are not step-by-step failure rates between successive steps of pursuing a solution. They are overall end-to-end success rates on extremely difficult problem sets that most humans would score close to zero on, and most humans with sufficient training and experience would score significantly less than 90%. And they are problem sets that grow successively harder as AIs get successively more capable. A few years ago AIs were being tested on high-school level math problems. Now they are being tested on problem sets that challenge people with PhDs. Or problem sets that humans are (so far) incapable of solving.
Compare that to humans. How often do you think humans get it right the first time? Almost never, for sufficiently complex problems. And how often do humans get the overall result wrong, without realizing it? Also, almost always for sufficiently complex problems, or sufficiently ordinary people. And what percent of the general public would be able to get even one question right in a problem set based on Math Olympiad problems? What percentage of mathematicians are capable of solving even one Millenium Problem?
Furthermore, you have to take into account that AIs do not particularly use output to input architectures. I frequently see self-corrections to plans when I'm developing using AI assistants, and plan changes are sometimes major. ("Oh I see. I need to revise step 4 of the plan." -- Claude paraphrased). Clearly current-generation AIs use more than input to output feedback; they also use supervisor/client strategies of AI and collaborative networks as well. And often provide self-correcting infrastructure, varying from "Claude: Oh. That generated a compiler error. Let's fix that.", to "Claude: One of the test cases is failing. Let's see what's going on.". And that self correcting infrastructure allows AIs to try again and again until they succeed. (Not unlike what humans do when solving difficult problems).
As a member of the 0.2% (but not the 0.00006%), I have to imagine that frontier models, when applied to research topics, are using most of those 10,000 instances to investigate alternate hypotheses in parallel. Which is also something that humans do across entire disciplines. Based on my personal experience with difficult technical and commercial and scientific problems, I sincerely believe that most breakthroughs are not made because whoever made them is more brilliant than the other hundred or thousand or dozen qualified people who are working on the same problem, but are made because they happened to be lucky enough to start with a somewhat random hypothesis that luckily turned out to be the right one. In case you were concerned that this is a sour grapes point of view, yes, our team was the one that came in first.
So even 90% may be a very big deal, for problems that human get right about 0.1% of the time. Or even for problems that AIs get right about 0.1% (but have enough instances available to pursue 10,000 hypotheses in parallel).
p.s. Ironically, it really looks like humans score significantly less than 90% on the "is 90% good" problem.
> The internal models are more eager, and there is the scale difference of being able to spin up 10.000 vs 10, but to get something valuable out requires skill.
Yeah, but the skill is knowing when it's talking nonsense.
Same deal with the recent stuff about vibe coded Photoshop clones. Yeah, they work, but you'd have to do Adobe-level market research to understand what bar the product needs to pass, and the vibe-coders didn't have that, so they saw an image editor and mistook it for "basically fine".
> Agents that produce good results 90% of the time vs 93% of the time are not enough of a difference. Multiplied too many times, that error rate is too high for scale to become "magic".
Sure. However, one of the ways we see things change with, e.g. METR time horizons is that agents fail (IIRC) 10% of the tasks of some (human) duration all the time, pass 10% of that (human) duration all the time, and the rest have a random success rate you can fit to a sigmoid. As the agents get better, they still pass 10% all the time, still fail 10% all the time, but it's a harder set of questions.
2.5 years ago, I was thinking "I know what's a fun idea for April fools day! A programming language written entirely in runes!", the LLM produced non-compiling code for the compiler. 1.5 years ago, I thought "I'll give that another go", it managed it trivially and wasn't even interesting enough to bother with as an April fools day joke, not even with the video of a Viking explaining it to you in Icelandic.
We should probably assume the companies are 1-2 models ahead of the public, at all times. The HuggingFace incident happened about 2 months before the public got models of comparable competence.
> vibe coded Photoshop clones. Yeah, they work [...]
I think what people (sometimes) miss is that building something is one part of the work, but keeping it running, developing and refining it is a whole different story. And I think all those vibe-coded software will not stand the test of time.
Photoshop is 36 years old now and it wouldn't be the most-used, most-established solution for professionals if Adobe didn't do great work over all that time.
Not really rooting for Adobe here, dislike their subscription models just as anybody else, but I think you'll have to recognize that they kept it running and never stopped.
Absolutely.
I'm… well, "enthusiastic" would be the wrong word, but perhaps "impressed"?… with LLMs-as-coders, but they're objectively poor at business decisions, and Adobe keeping their business running all these years is a thing that I don't think current LLMs could have done.
2 replies →
No doubt the unreleased models are better at the things that LLMs are good at (coding, hacking, math) than what's been released, but there is still zero sign of their intelligence becoming broader and more general, and knowing how these models work and are trained there is no reason to expect them to become much more general unless changes are made.
The marketing message being pushed by Anthropic (& also OpenAI) is that these narrow math/hacking/coding skills are representative of the model's intelligence, when that seems to be the exact opposite of the truth - these narrow human/super-human competencies are entirely atypical of the models more general capabilities.
> but there is still zero sign of their intelligence becoming broader and more general
Ahem, getting better at these things is the sign:
- you, two and a half years ago, https://news.ycombinator.com/item?id=39720601
2 replies →
The public has yet to produce a result of comparable impact to HuggingFace, to my knowledge. It's not really about the model, it's more about orchestration infrastructure. Seems like we've hit the point where you don't necessarily need a smarter bot, you just need an arbitrary number of them working together.