Sam Altman was saying we'll eventually need to build a dyson sphere. We'll see how far human society can withstand this kind of escalation. I'm guessing one more year before there are major fractures in the basic economic/social/political relationship that get so bad it actually prevents more buildout.
I actually think we're in a strange situation with AI compute.
Right now, we have models that are statistical models of language, with a world model and reasoning "falling out" of a lot of effort.
It's like we've made something that's a little bit intelligent, and now we're trying to amplify that trick to create something that's quite intelligent. And - don't get me wrong - it works.
But it's also super, super inefficient. We're having machines "think out loud" to compensate for the quality of their thought processes. We elongate the path to make up for the progress made on a given step.
I tink there's probably a much smarter way of doing things that will require qualitative architectural (and quite possibly hardware) innovations. Right now we're on the path to a Dyson sphere: that's probably not going to be necessary once we figure out a smarter way to think.
I agree. But I think you're missing that LLMs can internalise a lot of the thinking process in their layers without explicit CoT. That System 1-style reasoning is bounded depth computation but very, very broad. Yudkowsky called it "cached thoughts" and I think it's an incredibly important idea [1]. It's really stiking how the best LLMs don't even need to think where smaller LLMs do.
So as more thinking is cached in their weights through increased RL training, those weights are doing more useful work and the efficiency is increasing.
The whole of knowledge work is being automated. We've barely begun to see the GPU build out. This is just the start.
I'd imagine they're going to 10x this, maybe 100x this.
yeah, spacex's planned million satellite datacenter-in-space constellation doesn't seem as absurd.
Sam Altman was saying we'll eventually need to build a dyson sphere. We'll see how far human society can withstand this kind of escalation. I'm guessing one more year before there are major fractures in the basic economic/social/political relationship that get so bad it actually prevents more buildout.
I actually think we're in a strange situation with AI compute.
Right now, we have models that are statistical models of language, with a world model and reasoning "falling out" of a lot of effort.
It's like we've made something that's a little bit intelligent, and now we're trying to amplify that trick to create something that's quite intelligent. And - don't get me wrong - it works.
But it's also super, super inefficient. We're having machines "think out loud" to compensate for the quality of their thought processes. We elongate the path to make up for the progress made on a given step.
I tink there's probably a much smarter way of doing things that will require qualitative architectural (and quite possibly hardware) innovations. Right now we're on the path to a Dyson sphere: that's probably not going to be necessary once we figure out a smarter way to think.
I agree. But I think you're missing that LLMs can internalise a lot of the thinking process in their layers without explicit CoT. That System 1-style reasoning is bounded depth computation but very, very broad. Yudkowsky called it "cached thoughts" and I think it's an incredibly important idea [1]. It's really stiking how the best LLMs don't even need to think where smaller LLMs do.
So as more thinking is cached in their weights through increased RL training, those weights are doing more useful work and the efficiency is increasing.
[1] https://www.lesswrong.com/posts/2MD3NMLBPCqPfnfre/cached-tho...