Comment by ben_w
9 hours ago
> The internal models are more eager, and there is the scale difference of being able to spin up 10.000 vs 10, but to get something valuable out requires skill.
Yeah, but the skill is knowing when it's talking nonsense.
Same deal with the recent stuff about vibe coded Photoshop clones. Yeah, they work, but you'd have to do Adobe-level market research to understand what bar the product needs to pass, and the vibe-coders didn't have that, so they saw an image editor and mistook it for "basically fine".
> Agents that produce good results 90% of the time vs 93% of the time are not enough of a difference. Multiplied too many times, that error rate is too high for scale to become "magic".
Sure. However, one of the ways we see things change with, e.g. METR time horizons is that agents fail (IIRC) 10% of the tasks of some (human) duration all the time, pass 10% of that (human) duration all the time, and the rest have a random success rate you can fit to a sigmoid. As the agents get better, they still pass 10% all the time, still fail 10% all the time, but it's a harder set of questions.
2.5 years ago, I was thinking "I know what's a fun idea for April fools day! A programming language written entirely in runes!", the LLM produced non-compiling code for the compiler. 1.5 years ago, I thought "I'll give that another go", it managed it trivially and wasn't even interesting enough to bother with as an April fools day joke, not even with the video of a Viking explaining it to you in Icelandic.
We should probably assume the companies are 1-2 models ahead of the public, at all times. The HuggingFace incident happened about 2 months before the public got models of comparable competence.
> vibe coded Photoshop clones. Yeah, they work [...]
I think what people (sometimes) miss is that building something is one part of the work, but keeping it running, developing and refining it is a whole different story. And I think all those vibe-coded software will not stand the test of time.
Photoshop is 36 years old now and it wouldn't be the most-used, most-established solution for professionals if Adobe didn't do great work over all that time.
Not really rooting for Adobe here, dislike their subscription models just as anybody else, but I think you'll have to recognize that they kept it running and never stopped.
Absolutely.
I'm… well, "enthusiastic" would be the wrong word, but perhaps "impressed"?… with LLMs-as-coders, but they're objectively poor at business decisions, and Adobe keeping their business running all these years is a thing that I don't think current LLMs could have done.
LLMs don't need to focus on the business. They can make the ideal software for you. Who cares if it's not the perfect software for me? I can have my LLM do that. Especially if there's an open source base for me to start.
1 reply →
No doubt the unreleased models are better at the things that LLMs are good at (coding, hacking, math) than what's been released, but there is still zero sign of their intelligence becoming broader and more general, and knowing how these models work and are trained there is no reason to expect them to become much more general unless changes are made.
The marketing message being pushed by Anthropic (& also OpenAI) is that these narrow math/hacking/coding skills are representative of the model's intelligence, when that seems to be the exact opposite of the truth - these narrow human/super-human competencies are entirely atypical of the models more general capabilities.
> but there is still zero sign of their intelligence becoming broader and more general
Ahem, getting better at these things is the sign:
- you, two and a half years ago, https://news.ycombinator.com/item?id=39720601
Yes, I failed to anticipate environment/playground-based RL-training and how much difference that would make, although I think the broader point about limitations of lack of continual learning still stands, and I can't see these simulated environments ever become complex and/or varied enough to be any substitute for the need to learn on the job.
However, gotcha acknowledged, but my point was about the jaggedness of AI, and how huge the divide in capability is between the domains where RLVR works, primarily coding, hacking and math, and everything else. The success and scale of RLVR in areas such as coding only make this gap more noticeable.
1 reply →
The public has yet to produce a result of comparable impact to HuggingFace, to my knowledge. It's not really about the model, it's more about orchestration infrastructure. Seems like we've hit the point where you don't necessarily need a smarter bot, you just need an arbitrary number of them working together.