Comment by usernomdeguerre
1 day ago
I feel like if the gap were so monumental that he could just point to a monument of it, rather than hand-waving at "the vertical professionals".
Alternative interpretation of the numbers; there's only a subset of productive work that LLM's are 'capable' of doing at this point. That is, work that's verifiable by a machine or doesn't require verification. Thus, the cleavage points are along that axis and-- naturally-- most human beings don't have that kind of work.
> That is, work that's verifiable by a machine or doesn't require verification
That's where I'm at on the capabilities. LLMs have demolished software and mathematics because those have well-developed verification that's easier than doing the thing itself. Stuff that's more subjective they can produce good work, but with a significant error rate which makes it hard to let them loose on important client work without checking. Or rather, a number of companies are going through the cycle of trying that, having an expensive error, and reconsidering.
Or sometimes a company can simply pass the error cost onto consumers, who are gradually arming themselves with LLMs in response. We're going to see a productivity-sapping boom in disputes.
> demolished software and mathematics because those have well-developed verification that's easier than doing the thing itself
They are proving useful for the parts of those things that have that verification, but in neither case is that the whole of the field. In maths, for example, we have seen LLMs find answers to some interesting problems/conjectures, but haven't yet seen them pose any similarly interesting problems/conjectures themselves because the criteria for a "good conjecture" are unclear whereas the criteria for a resolution of an existing conjecture are usually supplied by the conjecture itself.
Did anyone ask? i.e. did anyone with enough background to be able to judge whether a conjecture is "good" just point it at the repo of solved problems and ask it to formulate new conjectures based on the results/proofs? Or ask it to propose new structures to study? Or to identify a previously unknown general technique/tool that it had used in a proof?
It was asked to do a specific thing, and then its brain turned off/was reallocated to something else entirely (e.g. CRUD programming or answering celebrity trivia). It doesn't just idly think in its downtime. If we want it to do something else, we need to ask.
1 reply →
lowering the cost and speeding up delivery will mature technology in other fields leading go the disruptive verification that ai needs
The swarm-based HF hack, and the observation that 40-50% of the compute at AI labs goes into training the next generation of agents, doesn't count?
We're all over here using agents to code projects that used to take weeks and feeling pretty good about it, and they're using agents to build the next generation of frontier models — an order of magnitude more complex problem...
Whenever someone finds out how to apply this level of agentic sophistication to a problem that actually matters to consumers, then you'll have that monument that everyone can see.
> Whenever someone finds out how to apply this level of agentic sophistication to a problem that actually matters to consumers
Isn't this a massive alarm bell? That the capability hasn't done much for consumers yet that isn't basically toys? Although I suppose the pitch was always B2B "you can automate away your expensive white collar workers" and it's not clear how widespread that is actually happening.
It did a lot for consumers who dare to use it. It didn't do much for consumers who wait for middleman businesses to offer AI-based services, but that is not because of AI, but existing business model of SaaS.
The businesses pay for tokens and try to deliver their usual enshittified experience, and nothing interesting gets created, because it cannot - enshittification extinguishes all power of AI. Using AI to full extent, in contrast, obviates most need for the business in the first place. So they're all struggling to design an AI powered product, in a world where those very products are already becoming subsumed by AI.
So normies wait for middleware companies to get their shit together or (much more likely) finally die, while people with software experience and eagerness to use AI just don't subscribe, because their favorite LLM handles that work just fine.
2 replies →
What is the incentive to try? Any such application, if profitable, would immediately be replicated. If one person can use agents to do something profitable, anyone can use agents to easily copy it. Hence you have Anthropic locking down their models and removing features customers can access, to try and protect their own slim advantage.
I think if you build a product that takes a lot of serious AI work to both build and run, it's going to be hard to replicate just like past "good products" were hard to replicate. It's just that the bar for what's "hard to replicate" is way higher than before.
And I thought that building your own castle and hoping your moat is wide enough was always a losing strategy in the end. We shall see how it ends this time.
2 replies →
With this stuff I immediately wish I was more economically literate. Bare with me. Let’s say one of these labs can use agents to replicate work otherwise done by humans. This proves that anyone with agents can eventually do this, unfortunately I don’t think its a moat like a platform (like facebook) provides, given that everyone is leapfrogging each-other constantly. Thus, it kind of becomes a commodity / low margin business?
Depending on the type of output, the value immediately (?) goes to the electricity cost of the LLMs and the cost of materials. So: more “efficient”, but without a big margin, right? So its cheaper for the workers that remain working, other workers are displaced, and capital owners now have a super slim margin business which can be instantly replicated at a place where material cost are lower (?).
Thus, it almost seems like “value add” by humans would in this simplified view be vastly diminished, and all that remains is resource wealth. Pop the materials in on one end and let the AI manufacture the product. Great news for the third world assuming the USA doesn't invade them.
Also, whose the consumer? Or are we going back to a strange medieval time where consumers are not of concern?
I don’t know what to think
> all that remains is resource wealth
Final vindication of the Georgists. All value flows back as "rent", which in the 21st century includes IP licensing.
> Great news for the third world
Not sure about this in a world where there isn't demand for human labour. Especially in the post-consumer world. Very difficult future to see. Possible that it will deliver enough capability and expertise to drastically improve things; possible that without skilled human review that fails catastrophically.
I'm reminded of the Houthis asking Claude to build them a missile guidance system. We're going to see the Toyota Hilux + .50 cal "technical" combo get some weird targeting system upgrades.
1 reply →
Electricity is important, even more important than AI (despite some rhetoric).
We don’t see electricity companies among the biggest companies in the world, except in China where they are a state owned monopoly… maybe a good analogy after all.
1 reply →
> Bare with me.
It depends; can you send pics first?
1 reply →
Maybe raw intelligence is not the bottleneck for what can be accomplished with the current data available in a form agents can access it?
> The swarm-based HF hack, and the observation that 40-50% of the compute at AI labs goes into training the next generation of agents, doesn't count?
Most peoples jobs do not involve hacking other services in large scale fuzzing/whatnot attacks.
> Whenever someone finds out how to apply this level of agentic sophistication to a problem that actually matters to consumers, then you'll have that monument that everyone can see.
It does not seem to me that tech industry or its investors care about consumers even a little bit. Consumer preferences dont matter once you have enough or raw economic power, I guess.
My take as someone one tier out of the groups seeing AI going vertical (so I only watch it go up on a high exponential): the escape of "AI can do only automatically verifiable work well" and "AI doesn't have judgement" is a mirage, that will shatter within the year. AI already has better judgement than many senior people in some tech domains, on top of better skills, and I see absolutely no reason to believe it won't be able to deliver end-to-end solutions with no human involvement, that are more correct, robust and efficient than they would be if humans were allowed to weigh in.
I also see no reason for that to not generalize into every field of human endeavor.
You don't see a monument to vertical because its construction is bottlenecked on the time it takes for companies to form around it and start executing. So it's few months out still.
This weekend I had some time with Opus 5.5 and GLM 5.3. I went over some findings in a library of Claude Mythos 5.1. These were minor, low-severity bug fixes and Mythos provided a PR to address them. Its PR was functionally correct. I still spent ~5 hours looping over them until I was really happy, even though a fully working PR already existed.
Some of the differences were:
- Mythos recommended salvaging damaged labels. I pushed back: data whose structure is broken shouldn’t be trusted
- My simpler more readable test caught a regression that Mythos’s more elaborate test missed
- Mythos allowed invalid values, citing real-world data that I could not reproduce, while I decided to follow the spec
- Mythos kept the old error types. I made every kind of malformed input raise the same error, and accepted a small, documented breaking change for anyone who relied on the old ones
Overall, I much preferred the final version of my PR to the initial draft Mythos provided. However, getting there involved quite a bit of time messing around with the code, as well as some back and forth with Opus (which was valuable).
This is a recent example that matches my overall experience working with current SOTA models (the ones mere mortals like me have access to).
TL;DR: I don’t think AI already has better judgment than (most) senior people. Obviously, that might change with future models. But that’s a hypothetical scenario. It might as well not change anytime soon.
Resonates exactly with my experience.
I'll add they also seem to make 'bad' judgement calls around which layer to implement something, often preferring downstream fixes/hacks/workarounds when upstream fixes would be better.
They also duplicate code lots rather than finding and reusing existing.
Both of these patterns lead to degradation and time wasting in complex code bases. Part of the behavioural adaptation I've made in using these tools is heading these off as much as possible at prompt-time
My prompts are often verbose to several paragraphs
6 replies →
That mostly matches my experience, but there is a caveat that you're not considering.
You don't know what good judgement calls the AI made along the way. Or rather you do but not in detail because those are the part you looked at that said, okay, this is good code and I should ignore it. Your judgment is biased because you're looking at the bad parts and not the good parts.
I think we are already at the point where code quality doesn't matter anymore because no human can keep up with the output. I don't like the code that these AI agents produce, but what do I care when I no longer have to read the code? As long as other AI agents can continue to understand it, it's good enough.
do you really want AI to make life and death decisions? do you really want that power concentrated in the hands of the few who own the AI systems?
Yes! When ever I read comments like this it always makes me think the poster has led a much more charmed life than I have. Every institution we have that makes major decisions is an impenetrable mass of bureaucracy that takes forever. Give me the quick automated decision every time
1 reply →
Last week I was dealing with a bug where we were triggering some behavior in a service in all requests when actually we really shouldn't be triggering this behavior for one of our callers. It just so happens that this caller sends a unique flag to the service.
The suggestion from frontier models was to have our service check for this flag and use that to decide whether to trigger this behavior, even though the flag had nothing to do with this behavior semantically. I'd expect this from a junior engineer. Correct now, but almost surely to cause a bug in the future as the codebase evolves.
These tools are incredible and can achieve things I would never have imagined five years ago. I have yet to see their judgement exceed mine.
>AI already has better judgement than many senior people in some tech domains, on top of better skills
It has better judgement in math and coding, which have infinitely generatable training data; if you ask it a domain-specific question about something like sales or marketing, its answer is often horrendous.
> AI already has better judgement than many senior people in some tech domains
How would you quantify this?
How do you quantify judgement?
My proxy metric is observing the growing ratio of "AI suggested, I haven't even thought of that" to "AI nonsense I had to correct" (where it didn't turn out later AI was actually right).
1 reply →
> "AI doesn't have judgement" is a mirage, that will shatter within the year. AI already has better judgement than many senior people in some tech domains
No offense, but if you noticed it has better judgment than you, consider that you might not be a "senior people in a tech domain"
Well my job title says otherwise.
Right, if they want to sell AI as being human level, outside of a couple of narrow domains (math & coding), then how about using AI to build something that actually demonstrates that?!
It seems that every time they try to use AI to do some real world task like manage a vending machine, then it falls flat on it's face.
The first time DeepBlue beat Kasparov, or AlphaGo beat Sedol, I think everyone got it - we're got a machine that is superhuman at Chess, or at Go. We didn't need to see it beat 100 other people to get it. Yet people at OpenAI think we are so dumb that we don't get that they have built a DeepBlue for Math, and that they need to show us another 100 examples to hammer that point home.
So, next time, OpenAI or Anthropic want to try to impress us with how smart their models are, forget the math, forget the three.js animations (we get that too), and release a model that can do something more useful and not just "here's DeepBlue beating contestant #100".
The first two points Andrej says are the cause of the funnel seem to be pretty much the "alternative explanation" you share.
Idk, there are entire categories there he doesn’t touch upon, which may be indicative of his own blind spot.
Coding - of course I use LLMs at this point, and as he says in the past year it has gone from “ok nice try robot” to “build me a BI tool”, and being pleased with the outcome.
Outside of coding my major use case is organising the chaos of a complicated life. I currently have it managing (including comms and form filling) the import/export process for the international move of a pet, managing pricing on my Airbnbs (dynamite - massive revenue boost) and giving me a weekly performance report, coordinating and corralling documentation for the humans for said international move, and being an extremely dogged advocate/personal assistant in a commercial dispute, which it has won.
The use cases are honestly exploding. I’m kinda at the point where next year’s multi jurisdictional tax returns are probably going to be “yo machine, file my taxes, let me check your work before I hit submit”. The year after next, without doubt.
And just to underscore the comprehension gap… I’ve been doing this stuff for much of this year now, and two days ago my wife ran to me in a panic saying “your computer has been hacked!”, as she happened to notice one of my agents filing Airbnb documentation with the U.K. government in chrome, and had no idea agents could drive a desktop.
>managing pricing on my Airbnbs
so, scamming your customers?
This assumes the worst of the OP and is absolutely a very poor comment. Do better.
I a common monument is: "I need to wash my car. The car wash is 100 meters away. Should I walk, or drive?"
The general public has heard of this, tried it on free tier AI, and uses it as an example of how dumb AI is.
Anthropics cheapest tier paid AI, on lowest effort, gets this right.
Sure, they fix these embarrassing things as fast as they get pointed out.
Today I gave a free tier AI a photo of a burning ship posted on Twitter, and asked it to confirm it was an oil tanker. It said no, it was a bulk grain carrier, and gave a specific ID. I pointed out that the bulk grain carrier it had identified was painted black, not red (like most liquid carriers) like the one in the photo (as well as other obvious differences).
That is just one example. The free tier models continue to make very dumb mistakes all the time. Maybe the top secret models, too dangerous for the public, are able to distinguish a black ship from a red one. Who knows.
Using the paid anthrotpic models daily, it has not been my experience that they are just fixing public examples of foolishness, but are generally getting smarter. For example I have my own private, obscure bike racing physics test I use that the newer low tier models pass now.
Before fable, the top tier models didn't pass it. Now even Haiku at low effort does.