Comment by LarsDu88
1 day ago
Honestly, whether you think burning current SOTA to hardware is an overinvestment risk depends on what your definition of intelligence is. If you think intelligence is something that can grow like height such that 18 months from now we will basically be bowing down to machine god giants that are running on B200s, then investing in ASICs is the wrong move. However, if you you subscribe to the (very reasonable view) that intelligence is more like a round ball that we are trying to make as spherical as possible (ala Francois Chollet's writings), then at some point the ball will be smooth enough for most people and many tasks.
It takes about 18 months to go through the design, verification, and manufacturing process if you move at breakneck pace. Design could probably be sped up.
About 18 months ago the top model was GPT-4o. Not great by today's standards, but still good enough for many tasks (certainly a big chunk of chatbot queries). The current SOTA covers far more use cases, but importantly at a level that surpasses many thresholds of utility.
I think part of this thinking is in reverse. What I mean by that is people are saying something like "Imagine if 18 months ago a company started to create an ASIC based on GPT-4o (or whatever was good at the time). By today they'd have a model that is not great by today's standard but good enough".
Maybe, but what is the shelf-life of that 18 month decision? Barely good enough today, when it launches, starts to get worse and worse every month going forward. You have to recuperate that investment on your depreciating asset.
And you are competing against anyone with the foresight to use a TPU instead, and the benefit from any new paper that finds how to distill, quantize or whatever better so their solution gets algorithmic boosts while you are locked in.
At a 50-100x speedup even a GPT-4o class model could perhaps compete with much newer models simply by thinking deeper, doing harness-controlled Ralph loops, etc. Sure, then it might be "only" ~2-5x faster, but, you wouldn't need to throw all the ASICs into the trash bin.
One could also imagine hybrid models, where part of the model is burned into ASICs and part of the model exists in VRAM/HBM2 so it can be updated.
I don't have enough low-level knowledge to evaluate the technical or economic feasibility of the above ideas, however.
For many, many tasks, you only need good enough results that accomplish a clearly-defined set of criteria. Smaller models that run 50x faster, in this case, could be far superior to a much slower model that meets the same criteria.
Sometimes you need speed; sometimes you need quality. There are very different use cases for each. For my own workflows, I sometimes want something very simple done ASAP; other workflows need "subjective" reasoning and careful crafting of responses.
Giving a literal monkey the ability to press more keys faster doesn't get a good joke from it.
A model will often come up with worse results given more cycles of compute, only because it will tailspin from second guesses, rethinking and literal flip-flopping on concepts.
--- edit: to those following the thread below... if you look at the comments from the account replying, it's pretty obviously a pro-China account and all replies are antagonistic against anything other than a total submission to the Chinese state. My responses are intentionally antagonistic as every point I've brought up is completely ignored in favor of insults, so yeah, I've been insulting back.
Said the OAI/Anthropic stockholder
5 replies →