Comment by quotemstr
1 day ago
> Cerebrus/Talas
They are fast, but they're still programmable accelerators, not a model burned into the gates.
1 day ago
> Cerebrus/Talas
They are fast, but they're still programmable accelerators, not a model burned into the gates.
Cerebras is a programmable accelerator, but Taalas does burn the model into the gates.
Ah, right. I misremembered. Does Taalas have a meaningful advantage here then?
Taalas claims significant cost, speed, and power differential with running on Nvidia (and similar) hardware. Their whole thesis is that with making the hardware cost less, run faster, and run with less power mixed with their claimed fast production cycle for new weights, companies would plan on purchasing and then relegating the other chips to different workloads at the next update cycle. And the chips have facilities for fine-tunes that can by dynamically loaded. The info I've seen has been very light on details, but if they can even get halfway where they are planning, it'll be a massive shift.
6x. Taalas has Llama3.1 8B running at 18000 tok/s. Cerebras advertised that model at 3000 tok/s.
8 replies →