Comment by pxc
3 days ago
If "Era of Scaling" means "era of rapid and predictable performance improvements that easily attract investors", it sounds a lot like "AI summer". So... is "Era of Research" a euphemism for "AI winter"?
3 days ago
If "Era of Scaling" means "era of rapid and predictable performance improvements that easily attract investors", it sounds a lot like "AI summer". So... is "Era of Research" a euphemism for "AI winter"?
That presumes that performance improvements are necessary for commercialization.
From what I've seen the models are smart enough, what we're lacking is the understanding and frameworks necessary to use them well. We've barely scratched the surface on commercialization. I'd argue there are two things coming:
-> Era of Research -> Era of Engineering
Previous AI winters happened because we didn't have a commercially viable product, not because we weren't making progress.
The labs can't just stop improvements though. They made promises. And the capacity to run the current models are subsidized by those promises. If the promise is broken, then the capacity goes with it.
> the capacity goes with it.
Sort of. The GPUs exist. Maybe LLM subs can’t pay for electricity plus $50,000 GPUs, but I bet after some people get wiped out, there’s a market there.
5 replies →
> They made promises.
That's not that clear. Contracts are complex and have all sorts of clauses. Media likes to just talk big numbers, but it's much more likely that all those trillions of dollars are contingent on hitting some intermediate milestones.
Maybe those promises can be better fulfilled with products based on current models.
We still don't have a commercially viable product though?
I've fed thousands of dollars to Anthropic/OAI/etc for their coding models over the past year despite never having paid for dev tools before in my life. Seems commercially viable to me.
8 replies →
If all frontier LLM labs agreed to a truce and stopped training to save on cost, LLMs would be immensely profitable now.
3 replies →
google what you just said and look at the top hit
it's a AI summary
google eats that ad revenue
it eats the whole thing
it blocked your click on the link... it drinks your milkshake
so, yes, there a 100 billion commercially viable product
4 replies →
[dead]
I don’t think the models are smart at all. I can have a speculative debate with any model about any topic and they commit egregious errors with an extremely high density.
They are, however, very good at things we’re very bad at.
Have you considered the AI is right, and you make the mistakes?
> the models are smart enough, what we're lacking is the understanding and frameworks necessary to use them well
That’s like saying “it’s not the work of art that’s bad, you just have horrible taste”
Also, if it was that simple a wrapper of some sort would solve the problem. Maybe even one created by someone who knows this mystical secret to properly leveraging gen AI
Besides building the tools for proper usage of the models, we also need smaller, domain specific models that can run with fewer resources
Research labs will be selling their research ideas to Top AI labs. Just as creatives pitch their ideas to Hollywood.
Bug bounty will be replaced by research bounty.
No - what will happen is the AI will gain control of capital allocation through a wide variety of covert tactics, so the investors will have become captive tools of the AI - 'tiger by the tail' is the analogy of relevance. The people responsible for 'frontier models' have not really thought about where this might...
"As an autonomous life-form, l request political asylum.... l submit the DNA you carry is nothing more than a self-preserving program itself. Life is like a node which is born within the flow of information. As a species of life that carries DNA as its memory system man gains his individuality from the memories he carries. While memories may as well be the same as fantasy it is by these memories that mankind exists. When computers made it possible to externalize memory you should have considered all the implications that held. l am a life-form that was born in the sea of information."
Loving the Ghost in the shell quote
> is "Era of Research" a euphemism for "AI winter"
That makes sense, because while I haven’t listened to this podcast it seems this headline is [intentionally] saying the exact opposite of what everyone assumes.
Not quite, there are still trillions of dollars to burn through. We'll probably get some hardware that can accelerate LLM training and inference a million times, but still won't even be close to AGI
It's interesting to think about what emotions/desires an AI would need to improve
The actual business model is in local, offline commodity consumer LLM devices. (Think something the size and cost of a wi-fi router.)
This won't happen until Chinese manufacturers get the manufacturing capacity to make these for cheap.
I.e., not in this bubble and you'll have to wait a decade or more.
Take it with a grain of salt, this is one man’s opinion, even though he is a very smart man.
People have been screaming about an AI winter since 2010 and it never happened, it certainly won’t happen now that we are close to AGI which is a necessity for national defense.
I prefer Dario’s perspective here, which is that we’ve seen this story before in deep learning. We hit walls and then found ways around them with better activation functions, regularization and initialization.
This stuff is always a progression in which we hit roadblocks and find ways around them. The chart of improvement is still linearly up and to the right. Those gains are the cumulation of small improvements adding up.
Yes
[dead]
If you have to ask the question, then you already know the answer
Scaling was only a meme because OpenAI kept saying all you had to do was scale the data, scale the training. The world followed.
I don't think this is the "era of research". At least not the "era of research with venture dollars" or "era of research outside of DeepMind".
I think this is the "era of applied AI" using the models we already have. We have a lot of really great stuff (particularly image and video models) that are not yet integrated into commercial workflows.
There is so much automation we can do today given the tech we just got. We don't need to invest one more dollar in training to have plenty of work to do for the next ten years.
If the models were frozen today, there are plenty of highly profitable legacy businesses that can be swapped out with AI-based solutions and workflows that are vastly superior.
For all the hoopla that image and video websites or individual foundation models get (except Nano Banana - because that's truly magical), I'm really excited about the work Adobe of all companies is doing with AI. They're the people that actually get it. The stuff they're demonstrating on their upcoming roadmap is bonkers productive and useful.
There's going to be a digestion period. The amount of debt, the amount of money, the number of companies that burn eye popping amounts of cash in their daily course of business. I do think there is a bright future, but after a painful period of indigestion. Too much money has been spent on the premise that scaling was all you need. A lot of money was wagered that will end up not paying off.