← Back to context

Comment by verdverm

10 hours ago

> You can’t show ROI to investors by cutting inference costs

You can as long as you can saturate hardware with profit that makes up for training costs (which are also in free fall).

Anthropic is most at risk since they haven't bought any hardware (afaik), however OpenAi is building custom silicon now. The rest of Mag7 are primarily hardware vendors

12 comments

verdverm

Reply
TSiege  10 hours ago

Any sources on training costs in free fall? My understanding was training costs are proportional to model size. Is it just that compute costs are falling?

  • verdverm  10 hours ago

    MiMo RL'd their latest models for $3.5M (million, not billion). It's hardware, software, and algos

    https://mimo.xiaomi.com/rl/

    Also look to the history of stable diffusion

    • lifeisloving  9 hours ago

      Training costs are not in free fall, and not every "post training" process is equal, Everyone does this differently. The costs certainly haven't gone down, to train a new 10T param model from scratch, like at all.

      You cannot point at Xiaomis tiny model and compare it to training a 10T param model. The levels of dunning krueger are getting out of hand.

      You also arent calculating the salaries of the people involved, of which there's only a few thousand worldwide in your hiring pool.

      5 replies →