Comment by lolinder
20 days ago
You can't compare Facebook with ChatGPT because the costs per user are in totally different orders of magnitude. One $5/mo VPS can serve the traffic of several hundred thousand Facebook users, while ChatGPT needs an array of GPUs per active user. They can optimize this somewhat, but never as much as Facebook can.
This means that they're stuck with more expensive monetization plans to cover their free tier loss leader, hence the $200/mo Pro subscription. And once you're charging that kind of price to try to make ends meet, you're ripe for disruption no matter how good your name recognition.
"ChatGPT needs an array of GPUs per active user" - nit: you're exaggerating by a few orders of magnitude.
First, queries from users can be combined and fed into servers in batches so that hundreds of queries can be concurrently served by a single node. Second, people aren't on and asking ChatGPT questions every second of every day. I'd guess the median is more like ~single digit queries per day. Assuming average response length of 100 tokens and throughput of 50 tok/s at batch size 50, that's 25 QPS or 2.1M queries per day, or 420k users served per node at 5 queries per user per day.
Now, a single 8xH100 node is a lot more expensive than $5/mo, so you're directionally correct there, but I'd wager you can segment your market aggressively and serve heavily distilled/quantized models (small enough to fit onto single commodity GPUs, or even CPUs) to your free tier. Finally, this is subject to Huang's Law, which says every 2 years the cost of the same performance will more than halve.
People said similar things about Facebook. "Oh their user growth might be amazing, but they're not making any money, it's not a real business."
But it turns out that with enough funding, you can prioritize growth over profit for a very long time. And with enough growth, you can raise unlimited funds before you get to that point. And going this route is smart and effective if you want to get to a $1T valuation in under a decade.
So yeah, ChatGPT's margins might not be as high as Facebook's. But it doesn't really matter at this point, they're in growth mode. What matters is whether or not they'll be able to turn their lead and their mindshare into massive profits eventually, and while we can speculate on that, it's far too early to definitively say the answer is no.
Like I said, that doesn't apply when money isn't free, and it doesn't apply when your marginal cost per user is so high.
Rather than getting into the nitty gritty details of monetization, when we ask ourselves if OpenAi can nail product like Facebook did (I guess) to become the next tech giant, I think we have to ask whether it's even possible when the tech industry is as established as it is.
You would think existing megacaps would be all over any new market if there is a profit to be made. Facebook's competition was basically other startups. That said, Google seems to be dropping the ball almost as bad as Yahoo.
But sure, if there's absolutely no way to make money from consumer AI then that will also make it hard for oai to win the game.