Comment by kennywinker
3 hours ago
Unless there are major improvements to how much hardware it takes to run a 1T model, this is deeply unrealistic. First because why release hardware that puts your biggest customers (data centers) out of business. Second because as I understand it the data centers have bought up all the high end chip production capacity for at least the next year and unless the bubble pops that'll continue for a while.
Because for the company that will actually do it, their biggest customers aren’t data centers they are iPhone owners.
First off the math doesn’t math. Datacenters are willing to pay $50k for a single high end GPU. If you have unlimited capacity, yeah sell millions for $100 a pop or $10 a pop or whatever the bom cost of a phone GPU would be - but if you have limited capacity, you’re gonna sell all of that to the customer who is willing to pay the most PER UNIT.
Second off, this doesn’t work from a power consumption standpoint. When I run qwen3.6-35b, a far smaller model than op is suggesting, power usage spikes to 150-200W during inference. To fit a 1T model in the palm of my hand, the amount of processing required doesn’t fit the amount of power available.
Now I’m not saying this will never happen - there are some great leads, e.g. burning models directly on to a chip - but op’s scenario is definitely not happening in two years. Maybe 5, a lot more likely 10, unless of course local ai is made illegal