← Back to context

Comment by mtrovo

4 days ago

30 cents per million tokens is wild.

The relevant question is how easy is to setup 56x B200s in a cluster, are we talking about a adhoc infra team job or clicking a button on AWS?

4 comments

mtrovo

Reply
copperheron42  4 days ago

Is $0.30 per million tokens that wild? Haiku 5.5 is $0.10 per million input and $0.50 per million output. And that's before you get onto the cheaper cache pricing. Surely a self hosted GLM isn't better than Haiku 5.5 at coding?

  • mtrovo  4 days ago

    This is in/out cost, caching being mentioned as virtually free. Haiku is $0.50 up to 100k context window, $2.50 afterwards which for coding is not a high bar to cross.

    Regarding model quality GLM 5.3 is past the good enough bar for most coding tasks. I wouldn't be surprised if the limiting adoption by companies is price not frontier model quality for most of their tasks.

    Besides that, since this is a bank, you can see how significant the requirements are to run sensitive workflows within your own infrastructure premises.