150k TPM limit on public endpoint means that it's likely unusable for many coding tasks. When we've tried Cerebras in the past, our problem has always been rates. We'd love to not deal with dedicated and to have access to a more flexible rate pool.
Even trying it out, it seems like our account has gotten moved to some limbo where we can no longer add billing information.
```
Billing access restricted
Self-serve billing is not available on Enterprise accounts. Please contact your team for further questions.
```
We have no team (they removed themself from our slack channel after we talked about rate limits). Perplexingly, none of this even shows up in the request, which gives:
```
{"message":"Model does not exist or you do not have access to it.","type":"not_found_error","param":"model","code":"model_not_found"}
```
When the error is really about billing.
I always want to like Cerebras, but I get the vibe that as a tokens in tokens out consumer you are not valued at all.
> 150k TPM limit on public endpoint means that it's likely unusable for many coding tasks.
I don't understand. How does that make it unusable? Is the limit shared by an entire team at once?
150,000 tokens per minute is a lot. You could start hitting that with a lot of concurrent requests in your session, but even throttled to 150k TPM it's still going to be faster than anything else you find.
I think the 128K context limit is the real ceiling. These models aren't amazing at long context, but once you account for a short input prompt, the input files, and headroom for a compaction summary, there isn't a lot left for the problem.
It's a limit on input tokens. So that's 3 50k requests per minute. At Cerebras speeds, that's about 5 seconds of usage per minute.
I was very excited last year for their coding plan but seeing a burst of requests pulse and then sitting there watching the cooldown reset is really not a great time.
Even though each individual request was fast, the sessions were only maybe 10% faster on wall clock time since there was so much waiting time.
What kind of coding tasks would you expect to hit that limit? In my setup, on a very large codebase, it takes each agent 3-4 minutes at minimum to go past 100k tokens.
(note it's 150k uncached tokens, the total limit is 450k/min)
in my last tests with cerebras for coding tasks, most large tasks or anything greenfield would hit token limits. note that smaller models and the gpt-oss-120b style models they used to run are very prone to overthinking, so individual turns may be 3-10k tokens of just thinking + input + output.
i don't think it's quite apples-to-apples to compare to a frontier model or even a k3. the odds of success (file compiles? read the right context?) are lower and thinking is longer.
Yeah, their public service isn't a serious/competitive offering. They don't have the capacity to serve all the customers who might want to use them at that speed. The public service exists so they get some users on OpenRouter, and that shows them as #1 on speed, which proves their tech is very fast, which gets them billions in hardware sales/licensing. If you have big enough pockets they can probably dedicate capacity to you. But for reliably fast small models you might want to rent some GPUs.
I was wondering whether this was any good for programming, but it is too fast for its own good. There is a limit of 450,000 tokens per minute. I hit this limit in about 90 seconds and burned through $1.10 while doing so. This is because cached tokens count towards the token limit.
For comparison, I ran the same task with DeepSeek-V4-Flash, which finished in 172 seconds and cost $0.024 with a final context window size of 55217 tokens, while Qwen3.8-27B was not even close to being done with a 64178 context window.
This is a very efficient way to burn your money, but I would not recommend it for programming.
On the positive side, I got a $5 signup bonus, so it wasn't my own money.
Cached tokens count towards the limit as well. For example, if your context window is 50,000 tokens, it takes 9 requests to reach that limit without generating a single token.
It would be great if they made their inference capacity for this model available via OpenRouter; the fastest provider on OpenRouter right now is at ~80tps https://openrouter.ai/qwen/qwen3.8-27b#providers
Just a couple days ago I learned about ninfer (https://github.com/Neroued/ninfer) and on RTX 5090 I can now get ~200 tok/s and over 400 tok/s on concurrent requests which is plenty fast for a local model of this strength.
Just tried it on a medium size coding/debug problem on an existing codebase, observations:
- Input doesn't look faster than other models, it spends a lot of time reading
Read about 5M tokens
- Output is awesome, super fast as you expect from the 1500t/sec I think that's correct
- Tool call is failing more than say DS4, which leads to time wasted on retries (complex tools like browser control for example)
- Shell commands are still somewhat of a bottleneck
The net effect is that I spend about the same time waiting, and I still need to read that output so, at least for coding, it actually reconciles me with the 100-200t/sec you can get on DS4 or the like. Maybe that's a good sweet spot after all and faster t/sec is not where the bottleneck is.
Also maybe my setup (OMP) doesn't do the cache correctly but that's a huge cost driver... so atm it's quite pricy
I used their Coding Plan for a few months. It is genuinely difficult to keep up with the models. The output is so fast. Qwen 3.8 27B is likely one of the strongest models they've hosted so far.
Edit: it looks like this is only available on a API token pricing. Does anyone know if they have rolled out prompt caching yet? It used to get pretty expensive for agentic coding tasks with no prompt caching.
> There is no additional fee for using prompt caching. Input tokens, whether served from the cache or processed fresh, are billed at the standard input token rate for the respective model.
Strongest model that they host on the public endpoint. They do a super fast version of GPT 5.6 Sol for OpenAI and have bigger open models on dedicated endpoints.
Last time I got one, I had to log into a Discord server and wait for "the drop" and IIRC Daniel Kim was giving them out based on who was there at the time. They were gone in less than a minute. This was ~8 months ago.
I just did a little anecdotal test. Had pi + cerebras review a recent commit and asked a few quick followups on it. Worked great.
The Cerebras session cost me $1.60 and took a total of 5.1 mins. I did get a few brief 429 rate limit errors in there. The p50 speed was 890 tok/s and 0.64s TTFT.
Using OpenRouter averages, that would've cost $0.29 (no cache discount at Cerebras!) and would've taken about 14.4 minutes.
So on this one short session, cerebras was 5.6x more expensive in exchange for being 2.8x faster. Or, another way, $1.32 buys back about 9 minutes of your time. Not a bad trade IMHO but the cache situation is a real bummer. The longer your session the more relatively expensive Cerebras gets. The "good" news is you're also limited by its short context window.
(Also, I used to be on the Cerebras coding plan and the support is pretty bad for end users. My guess is these public endpoints are really just product demos for potential enterprise customers.)
Noticed they are present in OpenRouter, but Qwen 3.8 is not there yet. Hopefully it'll get there soon.
For those who haven't noticed though, the context size they allow for Qwen is just 128k. Still interesting as a specialized sub-agent but not really well suited for long tasks.
I'm saddened that Gemma4 is replaced by Qwen 3.8 on PayGo plan. Gemma4 31B is not coding model but it is excellent at intent understanding and task execution used in agentic software. This just shows that real world dominant usage for llms so far is to code generate. And not to augment business products. They must had barely anyone using Gemma to remove it from that tier.
I have used their gemma 4 31b model through kagi and getting real instantaneous answers is absolutely crazy. A very different feeling and UX. Even if the model is smaller, there is definitely a use case for these. I was wondering if they would put the qwen 27b model, it sounds very interesting to try.
The thing I didn’t realize for a while is 27B is rather smart. As many (or more) activated parameters as the flash models of the universe that we know about. It reasons very well. It just doesn’t have a lot of knowledge.
They seem to have good enough general intelligence that missing knowledge is not that big thing. If you are able to have a proper [free search engine], they can do almost anything. Having own local search index about relevant stuff can help a lof if you don’t want to pay for search API.
Really an aside, but yesterday I got the Gemma-4-12b (128k context) to build it's first web app in the minimal Dark Software Factory I've been building for myself.
The question is whether Cerebras is available... I've been trying to get https://www.cerebras.ai/code for at least 1 year now. It's all sold out. Always. I once joined their Discord, waited for the drop, and it all sold out in seconds. I haven't had enough time to put my card details. Somebody recommended that I should put my card details in advance, lol.
The next time I hear about them I am laughing, because when I could enjoy these powers? How many years I should be sitting in a waitlist...
Qwen 3.8 27B is an exceptional model for coding and ranks as one of the best local models for coding....BUT in my head I am confused why a company that's IPO'd doesn't invest in RL'd super specialized, super-damn-fast models for very specific tasks - instead of giving us the OSS GPT model from what feels like 200 years ago
I have a self hosted Qwen 3.8 27B and I find it to be unusably bad. Using it agentically, it will spin around in circles on even small tasks talking to itself until it loses context and starts again. I even had it say "I've forgotten the users initial question"
I have a self hosted Qwen 3.8 27B and I find it unbelievably cracked and dedicated. It's at least credibly attempted everything I've thrown at it. Just today I had it write a toy compiler with a JIT backend just to test out a concept, and that was with 4-bit quantization and 8-bit KV cache. Something has to be going wrong with your deployment.
I want a Qwen 3.8 27B hosted locally but I don't quite have the RAM for it. And, I don't want to buy the RAM until I prove I can use it.
Yesterday I did have success with Gemma-4-12b with 128k context. It fits in my RAM and it's relatively fast on my hardware.
I had to give it prompts that are quite a bit different from the way I use foundation models, but I did get it to work quite well. I feel like I could learn it's differences and get good at using it for real work.
Do I understand their pricing correctly? This is $10 per month for a developer account PLUS you pay $1.49/M for output tokens and $0.99/M for input tokens on Qwen 3.8 27b with a 128k context?
EDIT: Or, maybe it's just token pricing, but $10 is the minimum? Maybe it's that.
Why do they only host small models rather than the 2.4T version? Is the I/O and interconnect between the wafers bad due to the limited beachfront relative to the massive size of the chip?
They can host larger models by pipelining it on multiple wafers. Each wafer stores one layer and N layers can serve an N * 44 gb model with N concurrency. The limitation would of course be inter-wafer I/O, which my comment was getting at. That's probably how they can serve bigger models like GPT 5.6 Sol [1].
Funnily enough the pricing isn't that much worse than on openrouter, where the best price at the moment is $0.24 in / $2.55 out, vs $1 / $1.5 on Cerebras.
Sure, 4x input , but cheaper output.
Though Cerebras doesn't have prompt caching, so not great for agentic workloads. (they do, but it doesn't affect the price.
Tokens are the new latest and greatest nonsensical shit on the planet. It's amusing. I can't wait to see the world in 1-2 years and the hilarity of looking back on this day.
I have been their user for more than year even used coding plans, though for normal coding the quota will definitely be a blocker if you are using opencode because rpm are bit less. Good for products/api though.
Having the choice is good as you can make a trade-off between speed, perf, and quality.
Until last year, people had a single AI god they believed in (mostly Anthropic stuff). Now we have power to make choices (open-weights, SOTA, speed-optimized, etc) the same way you do for system designs.
150k TPM limit on public endpoint means that it's likely unusable for many coding tasks. When we've tried Cerebras in the past, our problem has always been rates. We'd love to not deal with dedicated and to have access to a more flexible rate pool.
Even trying it out, it seems like our account has gotten moved to some limbo where we can no longer add billing information.
``` Billing access restricted Self-serve billing is not available on Enterprise accounts. Please contact your team for further questions. ```
We have no team (they removed themself from our slack channel after we talked about rate limits). Perplexingly, none of this even shows up in the request, which gives:
``` {"message":"Model does not exist or you do not have access to it.","type":"not_found_error","param":"model","code":"model_not_found"} ```
When the error is really about billing.
I always want to like Cerebras, but I get the vibe that as a tokens in tokens out consumer you are not valued at all.
> 150k TPM limit on public endpoint means that it's likely unusable for many coding tasks.
I don't understand. How does that make it unusable? Is the limit shared by an entire team at once?
150,000 tokens per minute is a lot. You could start hitting that with a lot of concurrent requests in your session, but even throttled to 150k TPM it's still going to be faster than anything else you find.
I think the 128K context limit is the real ceiling. These models aren't amazing at long context, but once you account for a short input prompt, the input files, and headroom for a compaction summary, there isn't a lot left for the problem.
It's a limit on input tokens. So that's 3 50k requests per minute. At Cerebras speeds, that's about 5 seconds of usage per minute.
I was very excited last year for their coding plan but seeing a burst of requests pulse and then sitting there watching the cooldown reset is really not a great time.
Even though each individual request was fast, the sessions were only maybe 10% faster on wall clock time since there was so much waiting time.
6 replies →
128k context is not a limit of the model, that's a limit of implementation:
"Context Length: 262,144 natively and extensible up to 1,000,000 tokens."
https://huggingface.co/Qwen/Qwen3.8-27B
1 reply →
150k tokens per minute at 1.5k tokens per second means you can have like 3 users concurrently and that's not a lot.
150k by account. At 1.5k a second you hit it very quickly.
5 replies →
Cerebras the tech is awesome, cerebras the company is a trainwreck
Is this the chatjimmy asic approach with a bigger model?
1 reply →
What kind of coding tasks would you expect to hit that limit? In my setup, on a very large codebase, it takes each agent 3-4 minutes at minimum to go past 100k tokens.
(note it's 150k uncached tokens, the total limit is 450k/min)
in my last tests with cerebras for coding tasks, most large tasks or anything greenfield would hit token limits. note that smaller models and the gpt-oss-120b style models they used to run are very prone to overthinking, so individual turns may be 3-10k tokens of just thinking + input + output.
i don't think it's quite apples-to-apples to compare to a frontier model or even a k3. the odds of success (file compiles? read the right context?) are lower and thinking is longer.
This was my experience a year ago on some other model they could run super fast. Routine coding tasks would hit the per-minute token limits.
Just the math there... 150k TPM... and 15k TPS means... you can run for 10 seconds every minute?
The basic math boggles the mind.
Not sure how the rate limiting works, but it's 1.5k TPS, not 15k, so you could run it for 100s/min, which seems good enough to me
4 replies →
Yeah, their public service isn't a serious/competitive offering. They don't have the capacity to serve all the customers who might want to use them at that speed. The public service exists so they get some users on OpenRouter, and that shows them as #1 on speed, which proves their tech is very fast, which gets them billions in hardware sales/licensing. If you have big enough pockets they can probably dedicate capacity to you. But for reliably fast small models you might want to rent some GPUs.
I was wondering whether this was any good for programming, but it is too fast for its own good. There is a limit of 450,000 tokens per minute. I hit this limit in about 90 seconds and burned through $1.10 while doing so. This is because cached tokens count towards the token limit.
For comparison, I ran the same task with DeepSeek-V4-Flash, which finished in 172 seconds and cost $0.024 with a final context window size of 55217 tokens, while Qwen3.8-27B was not even close to being done with a 64178 context window.
This is a very efficient way to burn your money, but I would not recommend it for programming.
On the positive side, I got a $5 signup bonus, so it wasn't my own money.
Without prompt caching this becomes more expensive than fable 5.1 after turn 50, assuming you start with 40k tokens and add 2k per turn.
> There is a limit of 450,000 tokens per minute. I hit this limit in about 90 seconds
I'm confused. If it's 1500t/s, isn't that only 90k per minute? How do you hit a 450k/minute limit?
Cached tokens count towards the limit as well. For example, if your context window is 50,000 tokens, it takes 9 requests to reach that limit without generating a single token.
2 replies →
Could this also be coming from the problem that Qwen3.8-27B's default mode being "extra-high reasoning level"?
It would be great if they made their inference capacity for this model available via OpenRouter; the fastest provider on OpenRouter right now is at ~80tps https://openrouter.ai/qwen/qwen3.8-27b#providers
They do appear to host other models on OpenRouter so maybe Qwen3.8 will be there soon: https://openrouter.ai/provider/cerebras
We're serving it around 150-200tok/s (uses our new speculative decoding implementation on a DFlash2 draft model).
https://mixlayer.com, LAUNCH-Q38-27B gets you $5 in credits if you want to kick the tires.
I tried in your playground and got 14.2 tok/s?
6 replies →
I don't see any kind of input cache discount listed on your pricing page. Do you offer that, or is all input priced the same?
any way to see the tok/s for all the models listed on your homepage? curious which has the best speed/quality tradeoff for me
This feels great
Just a couple days ago I learned about ninfer (https://github.com/Neroued/ninfer) and on RTX 5090 I can now get ~200 tok/s and over 400 tok/s on concurrent requests which is plenty fast for a local model of this strength.
dang only for certain nvidia GPUs, had my hopes up
Ok, I need to try that. I'm getting 45tok/s with vLLM on my 6000. >600tok/s concurrent, but 45tok/s single request.
can't second ninfer enough. amazing tech
Just tried it on a medium size coding/debug problem on an existing codebase, observations: - Input doesn't look faster than other models, it spends a lot of time reading Read about 5M tokens - Output is awesome, super fast as you expect from the 1500t/sec I think that's correct - Tool call is failing more than say DS4, which leads to time wasted on retries (complex tools like browser control for example) - Shell commands are still somewhat of a bottleneck
The net effect is that I spend about the same time waiting, and I still need to read that output so, at least for coding, it actually reconciles me with the 100-200t/sec you can get on DS4 or the like. Maybe that's a good sweet spot after all and faster t/sec is not where the bottleneck is.
Also maybe my setup (OMP) doesn't do the cache correctly but that's a huge cost driver... so atm it's quite pricy
> "Also maybe my setup (OMP) doesn't do the cache correctly but that's a huge cost driver... so atm it's quite pricy"
I don't believe Cerebras has a cached input pricing? They don't list one on the model page:
https://news.ycombinator.com/item?id=49554520#49555094 ("Input tokens, whether served from the cache or processed fresh, are billed at the standard input token rate")
lol yeah just saw that, yeah that makes it unusable I think at least for me.
I wonder if they will do that with sol ultrafast!
They have cache, but it costs the same indeed, no idea what the point of the cache is
1 reply →
I can't believe this situation has not improved in years. Is cerebras' main business selling the hardware, then?
Cerebras is super constrained on capacity right now, all the support is going to enterprise customers.
Maybe they’re gunning for speedy non-interactive pricing? Or its a limit of the technology or a business decision?
[dead]
I used their Coding Plan for a few months. It is genuinely difficult to keep up with the models. The output is so fast. Qwen 3.8 27B is likely one of the strongest models they've hosted so far.
Edit: it looks like this is only available on a API token pricing. Does anyone know if they have rolled out prompt caching yet? It used to get pretty expensive for agentic coding tasks with no prompt caching.
It appears that they do support Prompt Caching: https://inference-docs.cerebras.ai/capabilities/prompt-cachi...
It doesn't reduce the price though.
> How are cached tokens priced?
> There is no additional fee for using prompt caching. Input tokens, whether served from the cache or processed fresh, are billed at the standard input token rate for the respective model.
Well, talk about flipping the narrative.
4 replies →
Strongest model that they host on the public endpoint. They do a super fast version of GPT 5.6 Sol for OpenAI and have bigger open models on dedicated endpoints.
Agreed, but in our SAAS I can tell some UX will sky-rocket to next level with this
The coding plan is gone now right?
Last time I got one, I had to log into a Discord server and wait for "the drop" and IIRC Daniel Kim was giving them out based on who was there at the time. They were gone in less than a minute. This was ~8 months ago.
i believe they used to have monthly plan, what happened to that?
I just did a little anecdotal test. Had pi + cerebras review a recent commit and asked a few quick followups on it. Worked great.
The Cerebras session cost me $1.60 and took a total of 5.1 mins. I did get a few brief 429 rate limit errors in there. The p50 speed was 890 tok/s and 0.64s TTFT.
Using OpenRouter averages, that would've cost $0.29 (no cache discount at Cerebras!) and would've taken about 14.4 minutes.
So on this one short session, cerebras was 5.6x more expensive in exchange for being 2.8x faster. Or, another way, $1.32 buys back about 9 minutes of your time. Not a bad trade IMHO but the cache situation is a real bummer. The longer your session the more relatively expensive Cerebras gets. The "good" news is you're also limited by its short context window.
(Also, I used to be on the Cerebras coding plan and the support is pretty bad for end users. My guess is these public endpoints are really just product demos for potential enterprise customers.)
Thanks! Is there something about their platform that prevents caching? Or are they just not passing on the discount?
The session had a 91.4% cache hit rate. They just give zero discount.
Noticed they are present in OpenRouter, but Qwen 3.8 is not there yet. Hopefully it'll get there soon.
For those who haven't noticed though, the context size they allow for Qwen is just 128k. Still interesting as a specialized sub-agent but not really well suited for long tasks.
Great observation. That’s not enough context even for some one shot xhigh requests.
When I put Qwen3.8 27B xhigh towards adding scope proxying to the Guice library, it one shotted a great impl using 250k context before stopping.
Part of the greatness of the model is that it just keeps going until it gets a great result. 128k context is disappointing.
I'm saddened that Gemma4 is replaced by Qwen 3.8 on PayGo plan. Gemma4 31B is not coding model but it is excellent at intent understanding and task execution used in agentic software. This just shows that real world dominant usage for llms so far is to code generate. And not to augment business products. They must had barely anyone using Gemma to remove it from that tier.
I have used their gemma 4 31b model through kagi and getting real instantaneous answers is absolutely crazy. A very different feeling and UX. Even if the model is smaller, there is definitely a use case for these. I was wondering if they would put the qwen 27b model, it sounds very interesting to try.
The thing I didn’t realize for a while is 27B is rather smart. As many (or more) activated parameters as the flash models of the universe that we know about. It reasons very well. It just doesn’t have a lot of knowledge.
They seem to have good enough general intelligence that missing knowledge is not that big thing. If you are able to have a proper [free search engine], they can do almost anything. Having own local search index about relevant stuff can help a lof if you don’t want to pay for search API.
1 reply →
Really an aside, but yesterday I got the Gemma-4-12b (128k context) to build it's first web app in the minimal Dark Software Factory I've been building for myself.
https://joeldare.com/a-local-open-weight-model-builds-its-fi...
I really wish they had their customer support somewhere else than Discord, which seems to think I'm a bot and doesen't accept my email or phone numbe
discord support can fix such issues
If you need customer support to access customer support, something is wrong; no?
Discord is simply a liability.
The question is whether Cerebras is available... I've been trying to get https://www.cerebras.ai/code for at least 1 year now. It's all sold out. Always. I once joined their Discord, waited for the drop, and it all sold out in seconds. I haven't had enough time to put my card details. Somebody recommended that I should put my card details in advance, lol.
The next time I hear about them I am laughing, because when I could enjoy these powers? How many years I should be sitting in a waitlist...
Qwen 3.8 27B is an exceptional model for coding and ranks as one of the best local models for coding....BUT in my head I am confused why a company that's IPO'd doesn't invest in RL'd super specialized, super-damn-fast models for very specific tasks - instead of giving us the OSS GPT model from what feels like 200 years ago
almost of their business is hosting Sol ultra fast or whatever for OpenAI to use internally
Especially since they still serve Codex-Spark, which is dogshit.
I have a self hosted Qwen 3.8 27B and I find it to be unusably bad. Using it agentically, it will spin around in circles on even small tasks talking to itself until it loses context and starts again. I even had it say "I've forgotten the users initial question"
I have a self hosted Qwen 3.8 27B and I find it unbelievably cracked and dedicated. It's at least credibly attempted everything I've thrown at it. Just today I had it write a toy compiler with a JIT backend just to test out a concept, and that was with 4-bit quantization and 8-bit KV cache. Something has to be going wrong with your deployment.
I run it locally at q4_k_xl on a r9700 with kv cache bf16 and while it thinks a lot, it’s still fast enough to do the task.
This model had its knowledge replaced with reasoning ability. The chain of thought what makes this reasoning effective.
So this is why you need to let it think and don’t quantize the kv cache.
I want a Qwen 3.8 27B hosted locally but I don't quite have the RAM for it. And, I don't want to buy the RAM until I prove I can use it.
Yesterday I did have success with Gemma-4-12b with 128k context. It fits in my RAM and it's relatively fast on my hardware.
I had to give it prompts that are quite a bit different from the way I use foundation models, but I did get it to work quite well. I feel like I could learn it's differences and get good at using it for real work.
Do I understand their pricing correctly? This is $10 per month for a developer account PLUS you pay $1.49/M for output tokens and $0.99/M for input tokens on Qwen 3.8 27b with a 128k context?
EDIT: Or, maybe it's just token pricing, but $10 is the minimum? Maybe it's that.
https://www.cerebras.ai/pricing
No. You buy a minimum of $10 worth of credit, then use it at $1.49/M rate. There is no recurring charge.
There is a separate subscription based plan, which is sold out now.
Got it. But, they also charge the same for cached tokens, so that probably closes the gap on Foundation models quite a bit.
1 reply →
(Was anyone able to create an account just now? I tried but onboarding falls into a redirect loop)
(update: I got my answer. support@ replied and said my email domain is on their blacklist. It was just me (and I've resolved it)).
yeah - used sign in with google
Why do they only host small models rather than the 2.4T version? Is the I/O and interconnect between the wafers bad due to the limited beachfront relative to the massive size of the chip?
They make a giant inference chip. Their inference service is basically just advertising for their core value prop: hardware.
The CEO was on Gradient Dissent a couple years ago: https://www.youtube.com/watch?v=qNXebAQ6igs
The wafer only has space for 44 gb of sram. If they offload ram they lose the speedup of having everything on 1 chip (the whole point of cerebras).
They can host larger models by pipelining it on multiple wafers. Each wafer stores one layer and N layers can serve an N * 44 gb model with N concurrency. The limitation would of course be inter-wafer I/O, which my comment was getting at. That's probably how they can serve bigger models like GPT 5.6 Sol [1].
[1] https://www.cerebras.ai/blog/accelerating-gpt-5-6-sol-ultraf...
1 reply →
[dead]
Mostly economics I'm sure
Funnily enough the pricing isn't that much worse than on openrouter, where the best price at the moment is $0.24 in / $2.55 out, vs $1 / $1.5 on Cerebras.
Sure, 4x input , but cheaper output. Though Cerebras doesn't have prompt caching, so not great for agentic workloads. (they do, but it doesn't affect the price.
It is 15x more expensive. Openrouter usually charges like 1/4 for cached input.
Most of the cost for agentic coding is input tokens, you pay for the whole context at each tool call or message. Output tokens is just a small rate
Should be available in OpenCode once this lands: https://github.com/anomalyco/models.dev/pull/6199/changes
It's going to cost a fortune in opencode without prompt caching.
Tokens are the new latest and greatest nonsensical shit on the planet. It's amusing. I can't wait to see the world in 1-2 years and the hilarity of looking back on this day.
I have been their user for more than year even used coding plans, though for normal coding the quota will definitely be a blocker if you are using opencode because rpm are bit less. Good for products/api though.
Ut oh, might be down: "Unable to connect to the server. Please check your connection and try again." when sending a message to Qwen 3.8 27B.
At that speed it's too pricey for agentinc tasks.
The target audience is who needs raw speed.
Having the choice is good as you can make a trade-off between speed, perf, and quality.
Until last year, people had a single AI god they believed in (mostly Anthropic stuff). Now we have power to make choices (open-weights, SOTA, speed-optimized, etc) the same way you do for system designs.
It's not a criticism, I was really looking forward to trying out such a powerful model at this speed.
But I burn my 5$ allowance in 10 minutes ... and only because I was hitting rate limits, without it would probably be less than a minute.
1 reply →
What are the best benchmarks/leaderboards that compare task completion time between provider+model combos?
How many years until chips like this are available to consumers?
Many. Too lucrative for certain companies and even governments to allow that to happen
The idea of custom software on the fly is coming
used their Code product with GLM4.7. its fun but if the model is bad it just doesn’t do much useful.
Hope they add such models to Code too :)
Yeah GLM 4.7 is from another decade at the speed we're going
Normal people: tok/s or t/s
Psychopaths: tok/SEC
ok fair, caps lock kept ON /o\
I like tps
do you get reports on them?
[flagged]
[flagged]
Your brain can wash laundry and cook pasta, so there is still a long way to go
(requires additional fleshy bits sold separately)
regarding my brain, my mother might disagree on the laundry part.
Your brain updates itself constantly and maintains your whole body, LLMs are static.
Still, 1500tokens/s is indeed wild
If you read the reasoning trace for Qwen 3.8, it does a whole lot of "uh" and "But, wait..." too
You're absolutely right - filler words are genuinely load-bearing
At 1500 tps, "uh" is about 0.7 ms, instead of 200-300 ms for a human.
fyi this is an AI bot account