If Anthropic announced AGI tomorrow, how much better would that model be than Fable 5? It's looking like the road to AGI is gradual and moat-less. Models seem capable of improving other models, and even without illegal distillations many are nipping at the heels of Anthropic.
Yeah, I think we're learning that we overestimated the relevance of recursive self-improvement in a singularity/intelligence takeoff scenario. We thought that once an AI could start improving itself, it would cause an exponential, self-reinforcing intelligence explosion.
Turns out that scaling up compute is much more important and also limits the upper end of intelligence.
The bigger mistake is assuming it would be better at everything all at once.
Suppose it can do 80% of what the 20th percentile human can do. That's a huge advance and very useful, but it means there are still things it's not very good at. If any of those things is (or becomes) a bottleneck, you're not getting the hockey stick graph.
I'm currently writing a blog post about data centres in orbit, and my current conclusion is that even though they can build one, they definitely can't put 1 million up there and would have better things to do if they could.
AGI? Too loosely defined. They lack a lot of competences which humans recognise when we see them but find it hard to put into words; on the other hand what they can do they already do faster than any human (and have greater breadth than any single human, but this usually doesn't matter because "coder" and "economist" and "translator" gets solved in human teams by hiring three people).
I do not think current ML has the tools to solve for quality. But we know it's possible for a really mediocre intelligence to make human level intelligence, because evolution made us, so for me the question of AGI is more a practical one: is it affordable?
(I also think not at the present time, but that's an "I think" not "I am analyzing it carefully").
Maybe you missed the part where starlink / orbiting datacenters don't really have to even make money as long as they partially fund rocket launch tests.
Or maybe you don't take Elon seriously when he talks about Mars.
A new player beating Boeing to the ISS was once a pipe dream.
LEO constellations were once a pipe dream.
Launching thousands of satellites was once a pipe dream.
You should know that a) they are already running "AI" chips on their current sats. and b) they are already producing kW of power on orbit and have ~10k sats on orbit. You can watch Scott Manley's video on it, where he does some rough calculations and explains the overall architecture. There is nothing stopping them to do this, from an engineering perspective. If it makes commercial sense, that's another question, but 5-10-20 years in the future things might change there as well.
Microsoft tried to put datacenters into ocean [1] and then shelved the idea, because even that you have lower amount of failures, you still have failures and somebody has to go there and fix them. Which turns out to be problem.
And in ocean you don't have to solve for radiation nor cooling.
I don't think people's argument is that it's impossible to put data centers into space. The argument is that the downsides (radiation, cooling, maintenance, power) are so severe that it is pointless to do it at scale.
Sure, you can do it. I bet humans could fly to Mars if we invested a massive amount of resources into it. Why, though? That’s the “easy” if you throw sufficient amounts of money at problems over sufficient periods of time you can solve them.
If you don’t care about making any more from it. How exactly would datacenters in space would be more profitable than those on earth?
If just Elon was taking about data centers in space, you could take it with a grain of salt. But there are other serious players talking about it like Google and blue origin that it should be pretty clear it can't just be dismissed with "you didn't think about cooling!"
If Anthropic announced AGI tomorrow, how much better would that model be than Fable 5? It's looking like the road to AGI is gradual and moat-less. Models seem capable of improving other models, and even without illegal distillations many are nipping at the heels of Anthropic.
What is an "illegal" distillation? Terms of service are not laws, and clearly copyright laws are no barriers to developing AI models.
Yeah, I think we're learning that we overestimated the relevance of recursive self-improvement in a singularity/intelligence takeoff scenario. We thought that once an AI could start improving itself, it would cause an exponential, self-reinforcing intelligence explosion.
Turns out that scaling up compute is much more important and also limits the upper end of intelligence.
The bigger mistake is assuming it would be better at everything all at once.
Suppose it can do 80% of what the 20th percentile human can do. That's a huge advance and very useful, but it means there are still things it's not very good at. If any of those things is (or becomes) a bottleneck, you're not getting the hockey stick graph.
Why would the creator of AGI sell it to anyone, when they could keep it to themselves and corner dozens of markets?
If AGI = Data from Star Trek, it would be a huge leap. Frankly, anything less I wouldn’t consider as AGI.
'Reach AGI', the same way SpaceX will put data centers in orbit. A pipe dream.
I'm currently writing a blog post about data centres in orbit, and my current conclusion is that even though they can build one, they definitely can't put 1 million up there and would have better things to do if they could.
AGI? Too loosely defined. They lack a lot of competences which humans recognise when we see them but find it hard to put into words; on the other hand what they can do they already do faster than any human (and have greater breadth than any single human, but this usually doesn't matter because "coder" and "economist" and "translator" gets solved in human teams by hiring three people).
I do not think current ML has the tools to solve for quality. But we know it's possible for a really mediocre intelligence to make human level intelligence, because evolution made us, so for me the question of AGI is more a practical one: is it affordable?
(I also think not at the present time, but that's an "I think" not "I am analyzing it carefully").
Maybe you missed the part where starlink / orbiting datacenters don't really have to even make money as long as they partially fund rocket launch tests.
Or maybe you don't take Elon seriously when he talks about Mars.
4 replies →
> will put data centers in orbit. A pipe dream.
Cheap access to space was once a pipe dream.
Reusable boosters were once a pipe dream.
A new player beating Boeing to the ISS was once a pipe dream.
LEO constellations were once a pipe dream.
Launching thousands of satellites was once a pipe dream.
You should know that a) they are already running "AI" chips on their current sats. and b) they are already producing kW of power on orbit and have ~10k sats on orbit. You can watch Scott Manley's video on it, where he does some rough calculations and explains the overall architecture. There is nothing stopping them to do this, from an engineering perspective. If it makes commercial sense, that's another question, but 5-10-20 years in the future things might change there as well.
Microsoft tried to put datacenters into ocean [1] and then shelved the idea, because even that you have lower amount of failures, you still have failures and somebody has to go there and fix them. Which turns out to be problem.
And in ocean you don't have to solve for radiation nor cooling.
[1] https://www.tomshardware.com/desktops/servers/microsoft-shel...
I don't think people's argument is that it's impossible to put data centers into space. The argument is that the downsides (radiation, cooling, maintenance, power) are so severe that it is pointless to do it at scale.
5 replies →
Sure, you can do it. I bet humans could fly to Mars if we invested a massive amount of resources into it. Why, though? That’s the “easy” if you throw sufficient amounts of money at problems over sufficient periods of time you can solve them.
If you don’t care about making any more from it. How exactly would datacenters in space would be more profitable than those on earth?
If just Elon was taking about data centers in space, you could take it with a grain of salt. But there are other serious players talking about it like Google and blue origin that it should be pretty clear it can't just be dismissed with "you didn't think about cooling!"
2 replies →
> You can watch Scott Manley's video on it, where he does some rough calculations and explains the overall architecture.
I'm currently writing a blog post, and there's one big thing everyone, including Scott Manley, missed.
Once I realised it, I wondered what took me so long to spot this issue.
4 replies →
I think it's such a vague term. If you showed someone in 2010 what we have now they would say it's science fiction.
If you used a time machine to go back to 2021 and showed someone the best open source LLMs from 2026 they would surely say “yeah that’s AGI”