Comment by samayashar
2 days ago
The secret to success for OpenAI, Anthropic and labs is the vision that they saw 10 years back and kept working on it. We're in awe of how models like GPT-6 Astra and Claude Opus 5.5 are performing today, but it's important to understand that they've been working on this before we knew about AI.
The next big thing is Robots and some stealth company building today is going to be a trillion-dollar giant in few years time.
That’s just hindsight. People work on a lot of different things. Breakthoughs and cumulative improvements reaching a tipping point might happen or not.
Agreed, Quantum Computing is a nice example of this.
LLMs were also a nice example of this before 2018 or so, so I wouldn't necessarily count on quantum computing being a disappointment forever.
> The next big thing is Robots
IMHO, robots won't become a thing until they're running local AI, at which point they will become the thing that crushes humanity.
Why?
Even for domestic robots people don't care, see just about every cloud connected smart vacuum robot and the numerous data leaks
And for companies, most of them are fine using SaaS and cloud based hosting platforms, so why would it be any different for robotics?
Sure there is the latency argument, so at least something doing very short term decision making needs to run on device or at the edge, but for most task the end to end time is in the single didit seconds to minuites, so the extra 100ms or whatever that you get going out to some model in the cloud isn't a deal breaker.
They’ll crush us all economically way before physically. But I wouldn’t discount the later happening if we make it through the former
I assume the first versions will have local AI handling basic functions like balance and motion, and remote AI handling higher-level decisionmaking. I also agree that robots feel like something we should approach very cautiously, but of course we’re not.
I think as soon as we have GenAI robots, when you ask it to do the dishes, it will roll its eyes and complain that the detergent smells bad and corrodes its extensors, and you should do the dishes because you are the only one who eats, and besides its charge is down to 47% and it gets a processor-ache when it gets below 50%, and it didn't ASK to be purchased.
This feels like a retcon of OpenAI. Their vision from back then is unrecognizable because they had to completely pivot after they saw the value of transformers. They kicked off the LLM arms race in 2022 because they thought they were competing with Google.
Like fuzzy logic before it, this arm of AI development will most likely hit a wall, find where it is useful, and then become cheap and embedded all over to the point where we won't even recognize it as AI anymore.
Nvidia also had a long-term vision from which they are clearly reaping the rewards.
> they've been working on this before we knew about AI
Don't confuse AI with LLMs. "We" know about AI for a long time. We even have a term for when AI fails expectations, AI winters.
Right. If you're new to the area, it may seem like AI came out of nowhere in 2022. If you dig a little, you'll be amazed that it somehow came from nowhere in ~2012. If you dig even more, you realize there was a wave in the late 90s, early 2000s about "machine learning" (e.g. SVMs) and before it there was an 80s wave of both neural nets, agent models, and logic-based AI, probabilistic graphical models. Then you dig more and you realize AI originated from that Dartmouth workshop by Minksy and others in the 50s. Then you dig more and realize McCulloch and Pitts already modeled neural nets as little logic circuits in the 1940s. Then you realize the role of Shannon, Turing etc. Then you realize that computers actually arose in a milieu with a much more AI-shaped vision, cybernetics etc. than what we today think of as computing (PCs etc). And the precursors in the thought-formalization and mechanization trend in math and philosophy at the start of the 20th century. And even more back Leibniz's calculus ratiocinator and "calculemus!" slogan to settle debates by reducing argumentation to computation.
The point is, typically when something seems like it came out of nowhere, it just means you didn't dig deep enough. Ideas don't come at an instant, fully formed like Athene from Zeus' forehead. It's brick by brick, one twist on an existing idea and zeitgeist at a time.
I was digging into this recently with ChatGPT. I’ve loosely followed the progression of ML and NN over the past 20 years, but struggled to put it into context of where an LLM lives. The big inflection point was the 2017 Attention Is All You Need paper [1].
[1] https://en.wikipedia.org/wiki/Attention_Is_All_You_Need
9 replies →