Comment by nl
1 year ago
> You cannot just magically spin up a billion-dollar hardware company overnight with software, customers, sales channels and support, etc.
Not saying it is easy or to do it magically.
Just noting that Groq (founded by the TPU creator) did exactly this.
Yes, and now after years of doing that Groq is pivoting to being a cloud compute company, renting their hardware through an API exactly the same way Google does.
Building out your own vertically integrated offering with APIs is comparatively a lot simpler and significantly less risky in the grand scheme. For one thing, cloud APIs naturally benefit from the opex vs capex distinction that is often brought up here -- this is a big sales barrier, and thus a big risk. This is important because you can flush mid-8-figures down the toilet overnight for a single set of photomasks, so you are burning significant capital way before your foot is ever close to the proverbial door, much less inside it. You aren't going to make that money back selling single PCIe cards to enthusiastic nerds on Hacker News; you need big fish. Despite allusions to the contrary (people beating down your door to throw you bathtubs of money with no question), this isn't easy.
Another good example of verticality is the software. The difference in scope and scale between "Tools that we run" and "Tools you can run" is actually huge. Think about things like model choice -- it can be much easier to support things like new models when you are taking care of the whole pipeline and a complete offering, versus needing to support compiler and runtime tools that can compile arbitrary models for arbitrary setups. You can call it cutting corners, but there's a huge amount of tricky problems in this space and the time spent on procedural stuff ("I need to run your SDK on a 15 year old CentOS install!") is time not spent on the core product.
There are other architectural reasons for them to go this route that make sense. But I really need to stress here that a big and important one is that hardware is, in fact, a very difficult business even with a great product.
(Disclosure: I used to work at Groq back in 2022 before the Cloud Compute offering was available and LLMs were all the rage.)
I don't think renting out hardware is a bad model at all. Google spinning out their TPU work in this manner could be fine.
I think some (large) buyers will want on-prem and they have large enough budgets to make that worthwhile.
I don't think "sell individual TPUs to random people" is a great model. Most are better served by the cloud rental approach (although they might not think so themselves).
Isnt Groq pivoting to the IaaS/SaaS model because hardware channel sales is hard and its easier for everyone to just use an API?