Comment by jpadkins
4 hours ago
When model intelligence reliably hits 90%-95% of current day knowledge worker tasks, they are going to burn those weight to silicon and we will see another 10X improvement in price/performance frontier.
The dynamic GPU clusters will be used for the 5% of tasks, and pushing out the frontier. Also there will be a set of knowledge tasks that are not done today (because they are too difficult for most knowledge workers), that will start being done in the future.
Burning the weights into silicon would be many orders of magnitude increase, not just 10x. It's kind of crazy that this hockey stick the AI hype bros talk about seems more and more every day like it might be real
https://taalas.com/ has done it already for a wildly obsolete model. 14000 tokens per second.
https://chatjimmy.ai/ is their interactive. Tiny context, very dumb, but absurdly fast. Imagine this as a tool call for claude code for trivial changes - the tool call from the harness takes longer than the execution.
See Cerebras and Groq as well.
Wow! You weren't kidding,
I just tried it too and 14,098 tokens in .05 seconds, I barely blinked and it was done. There was no typing at all appearing on the screen. It just showed up.
https://chatjimmy.ai/chats/01dc66a4-4b1b-4dea-bb5f-926855e37...
1 reply →
Wow. This is absolutely wild. I didn't expect that.
If we get to anywhere near this speed for the equivalent of the current models... I don't even know what to think about that future.
1 reply →
Holy crap, I was not prepared for how fast it responded. I just wrote "Just wanted to see how fast you are! Can you write me a quick story about a tiger who lives inside a block of cheese the size of a house?"
I pressed Enter, and the response was instant.
> Generated in 0.037s • 14,205 tok/s
This is unbelievable.
13 replies →
And don't underestimate how much money Google, Microsoft, Amazon and Meta still have to spend on this tech.
Blocking Fable for sure made it very politicl a lot sooner than i expected it to happen.
and because China already has massive problems of getting access, they are pushing it on hardware too like what Huawai did without EUV.
It seems China is already able to do DUV a lot sooner than others expected.
> It seems China is already able to do DUV a lot sooner than others expected.
That's the media and in particular US KOLs of all sorts driving the wrong impression of China and other places. China and many other places for example have fast public transport that the US doesn't and can't even imagine today. They're not behind.
China's DUV still isn't that production grade (mass produce-able) so don't get that hyped up the wrong way (in a different direction).
The whole China-is-behind with tech and in particular semi wasn't that they can't. The truth is they spent decades in internal politics and corruption. That all got solved with the bans, so thank the bans! Jensen even said the bans were bad.
Yep it will be ASICs and DSPs all over again. Orders of magnitude changes.
So which shovels companies are the ones to watch for burnt in silicon models ?
1 reply →
Not an expert on this but wouldn’t this be possible with something similar to an FPGA?
My understanding is that for FPGA the issue is either it eats all your gates on internal memory if you interleave, or it takes forever to load everything between the SRAM on the board and the actual FPGA component over a bus, last time I looked into it.
Weights can be baked into silicone or programmed into hardware ala FPGA, but the context will always be dynamic.
High speed SRAM is where the $$$ is
What does that mean though? Like some kind of a ROM memory ?
Stacked ROM can, in theory, be a lot denser than anything that depends on a capacitor and refresh cycle.
I don't think it would be that difficult to manufacture compared to other process tech. HBM is really hard to do compared to other memory types.
> When model intelligence reliably hits 90%-95% of current day knowledge worker tasks, they are going to burn those weight to silicon
Google is already working on a similar idea but more "flexible".
Explain.
I'm not who you responded to and I don't have any info on Google. Nor can I explain in detail due to NDAs. But multiple major players are working on something along the lines of what the parent is alluding to.
The "edge" AI landscape (in particular, what you can do with ~5W) is going to be nuts in about 18 months.