Comment by halJordan
2 hours ago
Today's data centers are being built for yesterday's inference need. There's a persistent cult belief that ai hasnt found a niche or that companies havent proven utility or use cases or whatever. the demand for ai (internal to hyperscaler, and external for everyone else) simply dwarfs what is available.
Both Anthropic and OpenAI have been having major load issues though. Up until yesterday, OpenAI was serving at only 30t/s per default.
You are in violent agreement with the comment you replied to.