Comment by himata4113 11 hours ago The speed says otherwise. I think they're increasing costs since they want to start seeing ROI. 4 comments himata4113 Reply JanSt 11 hours ago Those are (mostly) new, faster TPU himata4113 11 hours ago latest TPU's appear to reach 800tok/s rather than the advertised 300tok/s. mgambati 7 hours ago They demoed today 8i running ate 1300 to 1600ish tokens per second. I imagine that is caused by having a single rack serving the model just for the demo. 1 reply →
JanSt 11 hours ago Those are (mostly) new, faster TPU himata4113 11 hours ago latest TPU's appear to reach 800tok/s rather than the advertised 300tok/s. mgambati 7 hours ago They demoed today 8i running ate 1300 to 1600ish tokens per second. I imagine that is caused by having a single rack serving the model just for the demo. 1 reply →
himata4113 11 hours ago latest TPU's appear to reach 800tok/s rather than the advertised 300tok/s. mgambati 7 hours ago They demoed today 8i running ate 1300 to 1600ish tokens per second. I imagine that is caused by having a single rack serving the model just for the demo. 1 reply →
mgambati 7 hours ago They demoed today 8i running ate 1300 to 1600ish tokens per second. I imagine that is caused by having a single rack serving the model just for the demo. 1 reply →
Those are (mostly) new, faster TPU
latest TPU's appear to reach 800tok/s rather than the advertised 300tok/s.
They demoed today 8i running ate 1300 to 1600ish tokens per second. I imagine that is caused by having a single rack serving the model just for the demo.
1 reply →