B300 is about 46% faster for one stream and 65% faster in aggregate. AMD wins only after Wafer divides throughput // selected cloud-rental prices: $2.50/GPU-hour for MI355X versus $6 for B300.
Benchmarks are unreproducible, power costs are missing, ROCm was patched... and on and on.
Sure, it can serve that particular model at that particular size more economically. Good for them... in particular.
This part sounds like AI assisted setting this up and benchmarking it:
>The fix was trivially simple: zero-pad the head count 12→16, run the fast kernel, and extract the real 12 heads from the output.
I've recently used a frontier AI (ChatGPT 5.6 Sol on ultra) to set up a much smaller local model, and the performance optimizations it introduced left the model totally incoherent. (The model just repeats a single character, etc.)
When I see a line like the one I just quoted, it leaves me wondering if the setup is still coherent like a stock install of Kimi K3 on supported hardware.
Did they run any benchmarks on it to see if it is still correct?
I have never used sol, but I regularly use Claude to set up and benchmark local models per task. It is very thorough and has always returned good setups. The only thing you have to do is make sure you point at the model card. It’s always incredulous that models exist after its training cutoff.
K3 does an ok job of setting up, but its config searching isn’t nearly as thorough and its will confidently tell you it’s found the best setup when it’s only turned a few knobs. It’s also not a good evaluator of its own output. It rates its work too highly and seems kind of defensive when benchmarking. Still a good check because it does find stuff, but open a fresh session and don’t tell it where the results came from.
What K3 does do more than any other model I’ve found is investigate folder structures. If I want to benchmark it and other models I have to move the testing methodology and any reference to other results out of the project folder bc Kimi is like an ls bloodhound. It will find them.
>and its will confidently tell you it’s found the best setup when it’s only turned a few knobs.
on one hand , yeah : a model being more aggressive towards exploration of the decision space is usually a good thing.
on the other hand : I think that it's up to the operator to set rigid test criteria to make these things actually work well in a repeatable fashion.
so in other words, i'm glad claude is doing a better job for the way you're prompting the thing, but as a spectator from afar these kind of operator complaints usually spring up from the use of weak, ambiguous, under-considered prompts.
similar with the parents' complaint; there should have been a testing criteria for legible output that got immediately flagged or failed by the larger model.
it's a very hard sell for me to think that " a a a a a a a " is accepted as a valid language output by any near-SOTA-large model without some real coercion.
hi I work at wafer. yes we ran benchmarks. for example our Kimi K3 is live on open router and in order to host there you have to run accuracy checks. like tau/gpqa.
and then u must pass test regarding thinking, coherency, and tool calling
How is the capex on 8 * MI354X even remotely justified at less than $10/h?
Even without the base system, power and every other expenses:
365d * 24h * $2.95 = $25842/a invoicable.
That doesn't add up within one year, that doesn't add up in three years and it is questionable that it brings in the money during the lifetime of the device?
Where do you see less than $10/h for 8 * MI354X? I can only find $2.50 for 1 * MI355X (lowest I can find for rent on other websites is $2.65, but maybe they got a better deal).
That can’t be real. Modal rents out an RTX 6000 Pro for more than that. Nvidia will rent your GPUs at a fixed cheap rate if you buy from them. Perhaps AMD has a different subsidy style program. Because those MI355X are going cheap here.
Wafer is making themselves synonymous with slop in the inference space. Exaggerated unfair comparisons in all their results, twitter hype posts with alarm emojis etc.
> $2.50/GPU-hr for the MI355X, $6.00 for the B300, and $4.25 for the B200.
This is not an accurate price comparison for real terms.
This is a company that launched a token subscription (WaferPass), before weeks later, rugpulling the plan for being unsustainable while simultaneously claiming they achieved incredible inference efficiency gains worthy of paying them mind.
I went the gpus.io website & it’s $2.95/hr right now, this is like comparing MSRP to actual market price. B300s are in demand & hence cost more, but these lazy editors at wafer.ai can't be bothered to do TCO on actual ownership nor share code to replicate their setups. Instead just relying on current market prices to win one row, which isnt even about per/$ on actual MSRPs.
I wish they wouldn't call them "open source models". They aren't open source. They didn't publish the training data. They didn't publish the tools they used to train the model.
They published the weights. It's an "open weight model", a term that it seems nearly everyone has agreed is appropriate. Why is this company not using it?
> They didn't publish the tools they used to train the model.
That's not totally true - for example Ziphu (Z.ai, developers of GLM) have published their Slime RL-training framework, developed together with Tsinghua University.
Yes, we all realize that "open weights" is more accurate than "open source", and anyways the source code would not be very interesting - it's the training data and methods that mostly define these models.
I can understand not wanting to call it open source if they don't give you the algorithm and software used for training, but wanting the training data too? That's completely different
Model training now is not a straight forwards process of input data -> run tools -> get model.
There's a whole lot of alchemy going on. We don't quite understand what works and what doesn't. Think of it like painting with water color and having to improvise very often.
The only advantage over water color is that we can revert to a working state.
The weights are the preferred form for modifying or integrating with other models. There is no obligation in open source to transitively open source all of the documentation / tools used to create the open source project.
>a term that it seems nearly everyone has agreed is appropriate
Models being considered open source even if the original training code / data is not released also is something almost everyone has agreed to be appropriate.
> The weights are the preferred form for modifying or integrating with other models
It's the 2nd time I hear this argument and I'm already fed up with it
Is it the preferred way only because training is expensive? It's like saying binaries are the preferred way of modifying program because you can't afford to have a fast enough machine to compile it yourself.
Most people don't have resources to compile their own browser, but what makes some browsers open is access to the source.
Maybe it's the preferred way because the people sharing models are themselves working with weights and no data?
The models are open source, it’s never been a requirement for someone to document the entire process of creating something to be “open source”. Just sharing the source, in this case, the weights, meets the criteria. Software engineering’s obsession with precise terms is actually one of the things I’m glad is dying with ai automation.
They’re not really even being precise. The relevant software freedom, from the FSF is []
The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this.
Ported to the model world, this is fulfilled by sharing the weights and implementation. There’s almost nothing that having the training data gets you (other than actually training it). The weights plus a reference implementation let you see all the states to study the behavior, and let you fine tune it to do your bidding (the abliteration etc). The freedom is satisfied.
Some might argue that without the training data you couldn’t do some classes of experiments to see how it works, say leave-one-out retraining. I’d argue things like that are not really about the model but about ML research or the class of models, which while interesting is not a free software pre-requisite.
The source is actually all training data, plus the software used for training, including some scripts or instructions to run the entire thing end to end on your own computer. That's what open source has always been about.
The weights are the output of a program, it's a binary. It's not source.
> Software engineering’s obsession with precise terms is actually one of the things I’m glad is dying with ai automation.
Yes. The next time you go to your doctor you should hope he's not being overly precise; or the engineer that builds the bridge; or the software engineer that implemented the embedded software in your insulin pump.
There is no precise thinking without precise terms.
If you do good work you should at least take the time to review the slop that details that work for slopiness. Otherwise it's hard to take it seriously. Especially the prefill section.
The B200 is being forcefully compared against something which is not gonna fit within it's memory in a single node & not much details about multi-node interconnectivity, disagg or not. As expected of a shoddy slop.
The only point it won is of cost per hour is one aggregation website for rentals, the premium a B300 commands against the $3/hr AMD chip which no provider has in abundance. Never bothered to do TCO of owning the hardware either.
To the B200’s defence, its numbers are somewhat deflated by the fact that it pays a cross-node all-reduce on the decode critical path (RoCE v2 at ~195 Gb/s) — it’s the only config here that spans two nodes
Another ad posing as "science".
It's basically a Wafer/AMD advertisement.
B300 is about 46% faster for one stream and 65% faster in aggregate. AMD wins only after Wafer divides throughput // selected cloud-rental prices: $2.50/GPU-hour for MI355X versus $6 for B300.
Benchmarks are unreproducible, power costs are missing, ROCm was patched... and on and on.
Sure, it can serve that particular model at that particular size more economically. Good for them... in particular.
Since we're (as a society) spending tens of billions of dollars to serve models like this, it's a pretty interesting advertisement.
This part sounds like AI assisted setting this up and benchmarking it:
>The fix was trivially simple: zero-pad the head count 12→16, run the fast kernel, and extract the real 12 heads from the output.
I've recently used a frontier AI (ChatGPT 5.6 Sol on ultra) to set up a much smaller local model, and the performance optimizations it introduced left the model totally incoherent. (The model just repeats a single character, etc.)
When I see a line like the one I just quoted, it leaves me wondering if the setup is still coherent like a stock install of Kimi K3 on supported hardware.
Did they run any benchmarks on it to see if it is still correct?
I have never used sol, but I regularly use Claude to set up and benchmark local models per task. It is very thorough and has always returned good setups. The only thing you have to do is make sure you point at the model card. It’s always incredulous that models exist after its training cutoff.
K3 does an ok job of setting up, but its config searching isn’t nearly as thorough and its will confidently tell you it’s found the best setup when it’s only turned a few knobs. It’s also not a good evaluator of its own output. It rates its work too highly and seems kind of defensive when benchmarking. Still a good check because it does find stuff, but open a fresh session and don’t tell it where the results came from.
What K3 does do more than any other model I’ve found is investigate folder structures. If I want to benchmark it and other models I have to move the testing methodology and any reference to other results out of the project folder bc Kimi is like an ls bloodhound. It will find them.
>and its will confidently tell you it’s found the best setup when it’s only turned a few knobs.
on one hand , yeah : a model being more aggressive towards exploration of the decision space is usually a good thing.
on the other hand : I think that it's up to the operator to set rigid test criteria to make these things actually work well in a repeatable fashion.
so in other words, i'm glad claude is doing a better job for the way you're prompting the thing, but as a spectator from afar these kind of operator complaints usually spring up from the use of weak, ambiguous, under-considered prompts.
similar with the parents' complaint; there should have been a testing criteria for legible output that got immediately flagged or failed by the larger model.
it's a very hard sell for me to think that " a a a a a a a " is accepted as a valid language output by any near-SOTA-large model without some real coercion.
hi I work at wafer. yes we ran benchmarks. for example our Kimi K3 is live on open router and in order to host there you have to run accuracy checks. like tau/gpqa.
and then u must pass test regarding thinking, coherency, and tool calling
How is the capex on 8 * MI354X even remotely justified at less than $10/h?
Even without the base system, power and every other expenses: 365d * 24h * $2.95 = $25842/a invoicable.
That doesn't add up within one year, that doesn't add up in three years and it is questionable that it brings in the money during the lifetime of the device?
Where do you see less than $10/h for 8 * MI354X? I can only find $2.50 for 1 * MI355X (lowest I can find for rent on other websites is $2.65, but maybe they got a better deal).
> How is the capex on 8 * MI354X even remotely justified at less than $10/h?
Is anyone actually renting them out that cheap? The very cheapest on-demand price I see online is $14, and most providers are a lot higher
That can’t be real. Modal rents out an RTX 6000 Pro for more than that. Nvidia will rent your GPUs at a fixed cheap rate if you buy from them. Perhaps AMD has a different subsidy style program. Because those MI355X are going cheap here.
Wafer is making themselves synonymous with slop in the inference space. Exaggerated unfair comparisons in all their results, twitter hype posts with alarm emojis etc.
> $2.50/GPU-hr for the MI355X, $6.00 for the B300, and $4.25 for the B200.
This is not an accurate price comparison for real terms.
This is a company that launched a token subscription (WaferPass), before weeks later, rugpulling the plan for being unsustainable while simultaneously claiming they achieved incredible inference efficiency gains worthy of paying them mind.
I went the gpus.io website & it’s $2.95/hr right now, this is like comparing MSRP to actual market price. B300s are in demand & hence cost more, but these lazy editors at wafer.ai can't be bothered to do TCO on actual ownership nor share code to replicate their setups. Instead just relying on current market prices to win one row, which isnt even about per/$ on actual MSRPs.
gpus.io shows tensorweave pricing at $2.95/hr. Tensorweave just shows "Talk to sales".
How does one even purchase an AMD MI355X?
Have $300k in disposible income?
add a few grand to get a new breaker panel and a new circuit put in to power it
I wish they wouldn't call them "open source models". They aren't open source. They didn't publish the training data. They didn't publish the tools they used to train the model.
They published the weights. It's an "open weight model", a term that it seems nearly everyone has agreed is appropriate. Why is this company not using it?
> They didn't publish the tools they used to train the model.
That's not totally true - for example Ziphu (Z.ai, developers of GLM) have published their Slime RL-training framework, developed together with Tsinghua University.
https://github.com/THUDM/slime
Also, if you read Moonshot's Kimi 3 report, it does give a lot of architectural detail and details on training.
https://github.com/MoonshotAI/Kimi-K3/blob/main/k3_tech_repo...
Some discussion on this by HuggingFace here:
https://www.youtube.com/watch?v=MW8-kqd2SD8
Yes, we all realize that "open weights" is more accurate than "open source", and anyways the source code would not be very interesting - it's the training data and methods that mostly define these models.
Sorry, where exactly is "data" in "source"?
I can understand not wanting to call it open source if they don't give you the algorithm and software used for training, but wanting the training data too? That's completely different
For LLMs the training data is the source of their weights.
You can‘t reproduce the LLM without the same data
11 replies →
This trope was valid maybe in 2022.
Model training now is not a straight forwards process of input data -> run tools -> get model.
There's a whole lot of alchemy going on. We don't quite understand what works and what doesn't. Think of it like painting with water color and having to improvise very often.
The only advantage over water color is that we can revert to a working state.
And since none of the source is provided to make the thing, it’s not open source.
Really, it’s no more open source than a free calculator. It’s free, and you can use it to no or great effect. But, you sure as hell can’t make one.
So calling it open source is without question, wrong.
1 reply →
If they open sourced the training data and code but you had to train the network yourself, would that be open source?
Which some companies have done! Nvidia, I believe, among others.
Yes, obviously.
24 replies →
Yes
Agreed, it's "open weights". Open source would be to publish the entire process so you could tweak it if you wanted.
The weights are the preferred form for modifying or integrating with other models. There is no obligation in open source to transitively open source all of the documentation / tools used to create the open source project.
>a term that it seems nearly everyone has agreed is appropriate
Models being considered open source even if the original training code / data is not released also is something almost everyone has agreed to be appropriate.
> The weights are the preferred form for modifying or integrating with other models
It's the 2nd time I hear this argument and I'm already fed up with it
Is it the preferred way only because training is expensive? It's like saying binaries are the preferred way of modifying program because you can't afford to have a fast enough machine to compile it yourself.
Most people don't have resources to compile their own browser, but what makes some browsers open is access to the source.
Maybe it's the preferred way because the people sharing models are themselves working with weights and no data?
1 reply →
> There is no obligation in open source to transitively open source all of the documentation / tools used to create the open source project.
Open source means open source code. Open weight means a binary file dump, not unlike an exe file. There is nothing open source about it.
Its like having a closed source text editor that censors certain words, and an open source text editor that censors certain words.
The latter can easily be recompiled, the former requires reverse engineering. Both may give you a license to use them freely.
6 replies →
The models are open source, it’s never been a requirement for someone to document the entire process of creating something to be “open source”. Just sharing the source, in this case, the weights, meets the criteria. Software engineering’s obsession with precise terms is actually one of the things I’m glad is dying with ai automation.
> obsession with precise terms
They’re not really even being precise. The relevant software freedom, from the FSF is []
Ported to the model world, this is fulfilled by sharing the weights and implementation. There’s almost nothing that having the training data gets you (other than actually training it). The weights plus a reference implementation let you see all the states to study the behavior, and let you fine tune it to do your bidding (the abliteration etc). The freedom is satisfied.
Some might argue that without the training data you couldn’t do some classes of experiments to see how it works, say leave-one-out retraining. I’d argue things like that are not really about the model but about ML research or the class of models, which while interesting is not a free software pre-requisite.
[] https://www.gnu.org/philosophy/free-sw.html#four-freedoms
1 reply →
The source is actually all training data, plus the software used for training, including some scripts or instructions to run the entire thing end to end on your own computer. That's what open source has always been about.
The weights are the output of a program, it's a binary. It's not source.
> Software engineering’s obsession with precise terms is actually one of the things I’m glad is dying with ai automation.
Yes. The next time you go to your doctor you should hope he's not being overly precise; or the engineer that builds the bridge; or the software engineer that implemented the embedded software in your insulin pump.
There is no precise thinking without precise terms.
Model weights are literally a compressed blob of binary data. The end result of a compilation process.
1 reply →
If you do good work you should at least take the time to review the slop that details that work for slopiness. Otherwise it's hard to take it seriously. Especially the prefill section.
People complaining about the slop - what about the atrocious text/bg contrast? Burning my eyes out faster than a B300 ever could
AI slop
Yeah I think they at least put some light effort into making it readable though. Obviously slop but not quite as bad as most slop articles.
There is barely any effort, a simple GPT5.6 sol pro query rips the post apart.
6 replies →
Lol, such a lazily written article by wafer.ai
GPUs. 8× MI355X (TP8) B300 (TP8+DCP8)
Decode tok/s per stream 118 tok/s 172 tok/s
Peak aggregate. 952 tok/s 1,568 tok/s
Peak aggregate per GPU 119 tok/s 196 tok/s
On every row the B300 beat the MI355X
The B200 is being forcefully compared against something which is not gonna fit within it's memory in a single node & not much details about multi-node interconnectivity, disagg or not. As expected of a shoddy slop.
The only point it won is of cost per hour is one aggregation website for rentals, the premium a B300 commands against the $3/hr AMD chip which no provider has in abundance. Never bothered to do TCO of owning the hardware either.
Did you see this section?
No mention of Infiniband or SPX lol
[flagged]
[flagged]
[flagged]