Comment by raspasov
6 months ago
Anyone who claims that a poorly definined concept, AGI, is right around the corner is most likely:
- trying to sell something
- high on their own stories
- high on exogenous compounds
- all of the above
LLMs are good at language. They are OK summarizers of text by design but not good at logic. Very poor at spatial reasoning and as a result poor at connecting concepts together.
Just ask any of the crown jewel LLM models "What's the biggest unsolved problem in the [insert any] field".
The usual result is a pop-science-level article but with ton of subtle yet critical mistakes! Even worse, the answer sounds profound on the surface. In reality, it's just crap.
They’re great at working with the lens on our reality that is our text output. They are not truth seekers, which is necessarily fundamental to every life form from worms to whales. If we get things wrong, we die. If they get them wrong, they earn 1000 generated tokens.
Why do you say that LLMs are not truth seekers? If I express an informational query not very well, the LLM will infer what I mean by it and address the possible well-posed information queries that I may have intended that I did not express well.
Can that not be considered truth-seeking, with the agent-environment boundary being the prompt box?
Right now you’re putting in unrequested effort to get to an answer. Nobody is driving you to do this, you’re motivated to get the answer. At some point you’ll be satisfied, or you might give up because you have other things you want to do, more.
An LLM is primarily trying to generate content. It’ll throw the best tokens in there but it won’t lose any sleep if they’re suboptimal. It just doesn’t seek. It won’t come back an hour later and say “you know, I was thinking…”
I had one frustrating conversation with ChatGPT where I kept asking it to remove a tie from a picture it generated. It kept saying “done, here’s the picture without the tie”, but the tie was still there. Repeatedly. Or it’ll generate a reference or number that is untrue but looks approximately correct. If you did that you’d be absolutely mortified and you’d never do it again. You’d feel shame and a deep desire to be seen as someone who does it properly. It doesn’t have any such drive. Zero fucks given, training finished months ago.
1 reply →
They are not intrinsically truth seekers, and any truth seeking behaviour is mostly tuned during the training process.
Unfortunately it also means it can be easily undone. E.g. just look at Grok in its current lobotomized version
7 replies →
They keep giving me incorrect answers to verifiable questions. They clearly don't 'seek' anything.
6 replies →
LLM are a compressed version of their training dataset with a text based interactive search function.
Yes, but you're missing their ability to interpolate across that dataset at retrieval time, which is what makes them extremely useful. Also, people are willing to invest a lot of money to keep building those datasets, until nearly everything of economic value is in there.
not everything of economic value is data retrieval
26 replies →
Because hypetrain.
Exactly I am so tired to hear about AI… And they are not even AI! I am also losing faith in this field when I see how much they all push so much hype and lies like this instead of being transparent. They are not AGIs not even AIs… For now they are only models and your definition is a good one
> they are not even AI!
From the "Artificial Intelligence" Wikipedia page: old-school Google search is AI, chess bots are AI, Google Maps is AI, the YouTube recommendation algorithm is AI...
LLMs are definitely AI.
LLMs are useful in that respect. As are media diffusion models. They've compressed the physics of light, the rules of composition, the structure of prose, the knowledge of the internet, etc. and made it infinitely remixable and accessible to laypersons.
AGI, on the other hand, should really stand for Aspirationally Grifting Investors.
Superintelligence is not around the corner. OpenAI knows this and is trying to become a hyperscaler / Mag7 company with the foothold they've established and the capital that they've raised. Despite that, they need a tremendous amount of additional capital to will themselves into becoming the next new Google. The best way to do that is to sell the idea of superintelligence.
AGI is a grift. We don't even have a definition for it.
I hate the "accessible to the layperson" argument.
People who couldn't do art before, still can't do art. Asking someone, or something else, to make a picture for you does not mean you created it.
And art was already accessible to anyone. If you couldn't draw something (because you never invested the time to learn the skill), then you could still pay someone else to paint it for you. We didn't call "commissioning a painting" as "being an artist", so what's different about "commissioning a painting from a robot?"
1 reply →
I an not an expert but I have a serious counterpoint.
While training LLMs to replicate the human output, the intelligence and understanding EMERGES in the internal layers.
It seems trivial to do unsupervised training on scientific data, for instance, such as star movements, and discover closed-form analytic models for their movements. Deriving Kepler’s laws and Newton’s equations should be fast and trivial, and by that afternoon you’d have much more profound models with 500+ variables which humans would struggle to understand but can explain the data.
AGI is what, Artificial General Intelligence? What exactly do we mean by general? Mark Twain said “we are all idiots, just on different subjects”. These LLMs are already better than 90% of humans at understanding any subject, in the sense of answering questions about that subject and carrying on meaningful and reasonable discussion. Yes occasionally they stumble or make a mistake, but overall it is very impressive.
And remember — if we care about practical outcomes - as soon as ONE model can do something, ALL COPIES OF IT CAN. So you can reliably get unlimited agents that are better than 90% of humans at understanding every subject. That is a very powerful baseline for replacing most jobs, isn’t it?
5 replies →
> Superintelligence is not around the corner. OpenAI knows this and is trying to become a hyperscaler / Mag7 company with the foothold they've established and the capital that they've raised.
+1 to this. I've often wondered why OpenAI is exploring so many different product ideas if they think AGI/ASI is less than a handful of years away. If you truly believe that, you would put all your resources behind that to increase the probability / pull-in the timelines even more. However, if you internally realized that AGI/ASI is much farther away, but that there is a technology overhang with lots of products possible on existing LLM tech, then you would build up a large applications effort with ambitions to join the Mag7.
LLMs require the sum total of human knowledge to ape what you can find on google, meanwhile Ramanujan achieved brilliant discoveries in mathematics using nothing but a grade school education and a few math books.
You phrase it as a diss but "Yeah LLM suck, they aren't even as smart as Ramanujan" sounds like a high praise to me.
5 replies →
yeah I've been thinking about them as stochastic content addressable memory. You can put as many next = userInput; while(true's) { next = mem[next]; } around them as you need in different forms. Single shot. Agents. etc and get wildly cool results out, but it's gated by some of the limitations there.
Thousands are being laid off, supposedly because they're "being replaced with AI," implying the AI is as good or better as humans at these jobs. Managers and execs are workers, too--so if the AI really is so good, surely they should recuse themselves and go live a peaceful life with the wealth they've accrued.
I don't know about you, but I can't imagine that ever happening. To me, that alone is a tip off that this tech, while amazing, can't live up to the hype in the long term.
Some employees can be replaced by AI. That part is true. It's not revolutionary (at least not yet) — it's pretty much the same as other post-industrial technologies that have automated some types of work in the past. It also takes time for industries to adapt to these changes. Replacing workers couldn't possibly happen in one year, even if our AI models were more far more capable than they are in practice
I'm afraid that what we're seeing instead are layoffs that are purely oriented at the stock market. As long as layoffs and talk about AI are seen as a positive signal for investors and as long as corporate leadership is judged by the direction the stock price goes, we will see layoffs (as well as separate hiring sprees for "AI Engineers").
It's a telltale sign that we're seeing a large number of layoffs in the tech sector. It is true that tech companies are poised to adapt AI more quickly than others but that doesn't seem to be what's happening. What seem to be happening is that tech companies have been overhiring throughout the decade leading up to the end of COVID-19. At that time hiring was a positive signal — now firing is.
I don't think these massive layoffs are good for tech companies in the long term, but since they mostly affect things that don't touch direct revenue generating operations, they won't hurt in the near-term and by the time company starts feeling the pain, the cause would be too long in the past to be remembered.
> Some employees can be replaced by AI.
Yes, but not lets pretend that there aren't a lot of middle and even upper management that couldn't also be replaced by AI.
Of course they won't be because they are the ones making the decisions.
3 replies →
I don't think anyone is being laid off because of AI. People are being laid off because the market is bad for a myriad of reasons, and companies are blaming AI because it helps them deflect worry that might lower their stock price.
Companies say "we've laid people off because we're using AI,x but they mean "we had to lay people off, were hoping we can make up for them with AI."
> I don't think anyone is being laid off because of AI.
I think that's demonstratively false. While many business leaders may be overstating it, there are some pretty clear cut cases of people losing their jobs to AI. Here are 2 articles from the Washington Post from 2 years ago:
https://archive.vn/C5syl "ChatGPT took their jobs. Now they walk dogs and fix air conditioners."
https://archive.vn/cFWmX "ChatGPT provided better customer service than his staff. He fired them."
The wave of layoffs started couple of years before the AI craze (ChatGPT).
> Managers and execs are workers, too--so if the AI really is so good, surely they should recuse themselves and go live a peaceful life
One thing that doesn't get mentioned is AI capability for being held accountable. AI is fundamentally unaccountable. Like the genie from the lamp, it will grant you the 3 wishes but you bear the consequences.
So what can we do when the tasks are critically important, like deciding on an investment or spending much time and resources on a pursuit? We still need the managers. We need humans for all tasks of consequence where risks are taken. Not because humans are smarter, but because we have skin.
Even on the other side, that of goals, desires, choosing problems to be solved - AI has nothing to say. It has no desires of its own. It needs humans to expose the problem space inside which AI could generate value. It generates no value of its own.
This second observation means AI value will not concentrate in the hands of a few, but instead will be widespread. It's no different than Linux, yes, it has a high initial development cost, but then it generates value in the application layer which is as distributed as it gets. Each human using Linux exposes their own problems to the software to get help, and value is distributed across all problem contexts.
I have come to think that generating the opportunity for AI to provide value, and then incurring the outcomes, good or bad, of that work, are fundamentally human and distributed across society.
> Thousands are being laid off, supposedly because they're "being replaced with AI," implying the AI is as good or better as humans at these jobs.
I don't think the "implying the AI is as good or better as humans" part is correct. While they may not be saying it loudly, I think most folks making these decisions around AI and staffing are quite clear that AI is not as good as human workers.
They do, however, think that in many cases it is "good enough". Just look at like 90%+ of the physical goods we buy these days. Most of them are almost designed to fall apart after a few years. I think it's almost exactly analogous to the situation with the Luddites (which is often falsely remembered as the Luddites being "anti-technology", when in reality they were just "pro-not-starving-to-death"). In that case, new mechanized looms greatly threatened the livelihood of skilled weavers. The quality of the fabric from these looms tended to be much worse than those of the skilled weavers. But it was still "good enough" for most people such that most consumers preferred the worse but much cheaper cloth.
It's the same thing with AI. It's not that execs think it's "as good as humans", it's that if AI costs X to do something, and the human costs 50X (which is a fair differential I think), execs think people will be willing to put up with a lot shittier quality if the can be delivered something much more cheaply.
One final note - in some cases people clearly do prefer the quality of AI. There was an article on HN recently discussing that folks preferred Waymo taxis, even though they're more expensive.
Not surprising people like Waymos even though they are a bit more expensive. For a few more dollars you get:
- arguably a very nice, clean car
- same, ahem, Driver and driving style
With the basic UberX it’s a crapshoot. Good drivers, wild drivers, open windows, no air-con. UberX Comfort is better but there’s still a range.
Every few weeks I give LLMs a chance to code something for me.
Friday I laid out a problem very cleanly. Take this datastructure and tranform it into this other datastructure in terraform. With examples of the data in both formats.
After the seventh round of back and forth where it would give me code that would not compile or code that gave me a totally different datastructure, giving it more examples and clarifications all the while I gave up. I gave the problem to a junior and they came back with the answer in about an hour.
Next time an AI bro tells you that AI can 'replace your juniors' tell him to go to hell.
This is a good summary of what LLM offer today.
My company is desperately trying to incorporate AI (to tell investors they are). The fact that LLM gets thing wrong is a huge problem since most work can’t be wrong and if if a human needs to carefully go through output to check it, it’s often just as much work as having that same human just create the output themselves.
But languages is one place LLMs shine. We often need to translate technical docs to layman language and LLMs work great. It quickly find words and phrases to describe complex topics. Then a human can do a final round of revisions.
But anything de novo? Or requiring logic? It works about as well as a high school student with no background knowledge.
Fundamentally, they are really powerful text transformers with some additional capability. The further away from that sweet spot and the closer to anthropomorphization the more unreliable the output
Interesting. I think the key to what you wrote is "poorly definined".
I find LLMs to be generally intelligent. So I feel like "we are already there" -- by some definition of AGI. At least how I think of it.
Maybe a lot of people think of AGI as "superhuman". And by that definition, we are not there -- and may not get there.
But, for me, we are already at the era of AGI.
I would call them "generally applicable". "intelligence" definitely implies leaning - and I'm not sure RAG, fine-tuning, or 6monthly updates counts - to split hairs.
Where I will say we have a massive gap, which makes the average person not consider it AGI, is in context. I can give a person my very modest codebase, and ask for a change, and they'll deliver - mostly coherently - to that style, files in the right place etc. Still to today with AI, I get inconsistent design, files in random spots, etc.
> I find LLMs to be generally intelligent. So I feel like "we are already there" -- by some definition of AGI. At least how I think of it.
I don't disagree - they are useful in many cases and exhibit human like (or better) performance in many tasks. However they cannot simply be a "drop in white collar worker" yet, they are too jagged and unreliable, don't have a real memory etc. Their economic impact is still very much limited. I think this is what many people mean when they say AGI - something with a cognitive performance so good it equals or beats humans in the real world, at their jobs - not at some benchmark.
One could ask - does it matter ? Why can't we say the current tools are great task solvers and call it AGI even if they are bad agents? It's a lengthy discussion to have but I think that ultimately yes, agentic reliability really matters.
that's the thing about language. we all kinda gotta agree on the meanings
I agree with the last part but I think that criticism applies to many humans too so I don't find it compelling at all.
I also think by original definition (better than median human at almost all task) it's close and I think in the next 5 years it will be competitive with professionals at all tasks which are nonphysical (physical could be 5-10 years idk). I could be high on my own stories but not the rest.
LLMs are good at language yes but I think to be good at language requires some level of intelligence. I find this notion that they are bad at spatial reasoning extremely flawed. They are much better than all previous models, some of which are designed for spatial reasoning. Are they worse than humans? Yes but just the fact that you can put newer models on robots and they just work means that they are quite good by AI standards and rapidly improving.
I'll offer a definition of AGI:
An AI (a computer program) that is better at [almost] any task than 5% of the human specialists in that field has achieved AGI.
Or, stated another way, if 5% of humans are incapable of performing any intellectual job better than an AI can, then that AI has achieved AGI.
Note, I am not saying that an AI that is better than humans at one particular thing has achieved AGI, because it is not "general". I'm saying that if a single AI is better at all intellectual tasks than some humans, the AI has achieved AGI.
The 5th percentile of humans deserves the label of "intelligent", even if they are not the most intelligent, (I'd say all humans deserve the label "intelligent") and if an AI is able to perform all intellectual tasks better than such a person, the AI has achieved AGI.
I think your definition is flawed.
Take the Artificial out of AGI. What is GI, and do the majority of humans have it? If so, then why is your definition of AGI far stricter than the definition of Human GI?
My definition is a high-bar that is undeniably AGI. My personal opinion is that there are some lower-bars that are also AGI. I actually think it's fair to call LLMs from GPT3 onward AGI.
But, when it comes to the lower-bars, we can spend a lot of time arguing over the definition of a single term, which isn't especially helpful.
1 reply →
I like where this is going.
However, it's not sufficient. The actual tasks have to be written down, tests constructed, and the specialists tested.
A subset of this has been done with some rigor and AI/computers have surpassed this threshold for some tests. Some have then responded by saying that it isn't AGI, and that the tasks aren't sufficiently measuring of "intelligence" or some other word, and that more tests are warranted.
You're saying we need to write down all intellectual tasks? How would that help?
If an AI is better at some tasks (that happen to be written down), it doesn't mean it is better at all tasks.
Actually, I'd lower my threshold even further--I originally said 50%, then 20%, then 5%--but now I'll say if an AI is better than 0.1% of people at all intellectual tasks, then it is AGI, because it is "general" (being able to do all intellectual tasks), and it is "intelligent" (a label we ascribe to all humans).
But the AGI has to be better at all (not just some) intellectual tasks.
2 replies →
I think any task-based assessment of intelligence is missing the mark. Highly intelligent people are not considered smart just because they can accomplish tasks.
I don't understand, you'll have to give an example.
What is the most non-task-like thing that highly intelligent people do as a sign of their intelligence?
3 replies →
Definitions around AI have been changing since the beginning, making it always farther in the future. In this system it can always be "right around the corner" but never arrive.
There's definitely also people in the futurism and/or doom and gloom camps with absolutely no skin in the game that can't resist this topic.
Where does Eric Schmidt fit? Selling something?
I think he's generally optimistic which is a net positive.
Why is that a net positive?
Already invested in the AI companies selling you something.
Its right around the corner when you prove it as fact. Otherwise as suggested it is just hype to sell us on your LLM flavor.
Alright, let’s get this straight.
You’ve got people foaming at the mouth anytime someone mentions AGI, like it’s some kind of cult prophecy. “Oh it’s poorly defined, it’s not around the corner, everyone talking about it is selling snake oil.” Give me a break. You don’t need a perfect definition to recognize that something big is happening. You just need eyes, ears, and a functioning brain stem.
Who cares if AGI isn’t five minutes away. That’s not the point. The point is we’ve built the closest thing to a machine that actually gets what we’re saying. That alone is insane. You type in a paragraph about your childhood trauma and it gives you back something more coherent than your therapist. You ask it to summarize a court ruling and it doesn’t need to check Wikipedia first. It remembers context. It adjusts to tone. It knows when you’re being sarcastic. You think that’s just “autocomplete”? That’s not autocomplete, that’s comprehension.
And the logic complaints, yeah, it screws up sometimes. So do you. So does your GPS, your doctor, your brain when you’re tired. You want flawless logic? Go build a calculator and stay out of adult conversations. This thing is learning from trillions of words and still does better than half the blowhards on HN. It doesn’t need to be perfect. It needs to be useful, and it already is.
And don’t give me that “it sounds profound but it’s really just crap” line. That’s 90 percent of academia. That’s every selfhelp book, every political speech, every guy with a podcast and a ring light. If sounding smarter than you while being wrong disqualifies a thing, then we better shut down half the planet.
Look, you’re not mad because it’s dumb. You’re mad because it’s not that dumb. It’s close. Close enough to feel threatening. Close enough to replace people who’ve been coasting on sounding smart instead of actually being smart. That’s what this is really about. Ego. Fear. Control.
So yeah, maybe it’s not AGI yet. But it’s smarter than the guy next to you at work. And he’s got a pension.
Something big is definitely happening but it's not the intelligence explosion utopia that the AI companies are promising.
> Who cares if AGI isn’t five minutes away. That’s not the point. The point is we’ve built the closest thing to a machine that actually gets what we’re saying. That alone is insane. You type in a paragraph about your childhood trauma and it gives you back something more coherent than your therapist. You ask it to summarize a court ruling and it doesn’t need to check Wikipedia first. It remembers context. It adjusts to tone. It knows when you’re being sarcastic. You think that’s just “autocomplete”? That’s not autocomplete, that’s comprehension
My experience with LLMs have been all over the place. They're insanely good at comprehending language. As a side effect, they're also decent at comprehending complicated concepts like math or programming since most of human knowledge is embedded in language. This does not mean they have a thorough understanding of those concepts. It is very easy to trip them up. They also fail in ways that are not obvious to people who aren't experts on whatever is the subject of its output.
> And the logic complaints, yeah, it screws up sometimes. So do you. So does your GPS, your doctor, your brain when you’re tired. You want flawless logic? Go build a calculator and stay out of adult conversations. This thing is learning from trillions of words and still does better than half the blowhards on HN. It doesn’t need to be perfect. It needs to be useful, and it already is.
I feel like this is handwaving away the shortcomings a bit too much. It does not screw up in the same way humans do. Not even close. Besides, I think computers should rightfully be held up to a higher standard. We already have programs that can automate tasks that human brains would find challenging and tedious to do. Surely the next frontier is something with the speed and accuracy of a computer while also having the adaptability of human reasoning.
I don't feel threatened by LLMs. I definitely feel threatened by some of the absurd amount of money being put into them though. I think most of us here will be feeling some pain if a correction happens.
I find it kind of funny that in order to talk to AI people, you need to preface your paragraph with "I find current AI amazing, but...". It's like, you guess it, pre-prompting them for better acceptance.
2 replies →
You say LLMs are “insanely good” at comprehending language, but then immediately pull back like it’s some kind of fluke. “Well yeah, it looks like it understands, but it doesn’t really understand.” What does that even mean? Do you think your average person walking around fully understands everything they say? Half of the people you know are just repeating crap they heard from someone else. You ask them to explain it and they fold like a cheap tent. But we still count them as sentient.
Then you say it’s easy to trip them up. Of course it is. You know what else is easy to trip up? People. Ask someone to do long division without a calculator. Ask a junior dev to write a recursive function that doesn’t melt the stack. Mistakes aren’t proof of stupidity. They’re proof of limits. And everything has limits. LLMs don’t need to be flawless. They need to be better than the tool they’re replacing. And in a lot of cases, they already are.
Now this part: “computers should be held to a higher standard.” Why? Says who? If your standard is perfection, then nothing makes the cut. Not the car, not your phone, not your microwave. We use tools because they’re better than doing it by hand, not because they’re infallible gods of logic. You want perfection? Go yell at the compiler, not the language model.
And then, this one really gets me, you say “surely the next frontier is a computer with the accuracy of a machine and the reasoning of a human.” No kidding. That’s the whole point. That’s literally the road we’re on. But instead of acknowledging that we’re halfway there, you’re throwing a tantrum because we didn’t teleport straight to the finish line. It’s like yelling at the Wright brothers because their plane couldn’t fly to Paris.
As for the money... of course there's a flood of it. That’s how innovation happens. Capital flows to power. If you’re worried about a correction, fine. But don’t confuse financial hype with technical stagnation. The tools are getting better. Fast. Whether the market overheats is a separate issue.
You say you're not threatened by LLMs. That’s cute. You’re writing paragraphs trying to prove why they’re not that smart while admitting they’re already better at language than most people. If you’re not threatened, you’re sure spending a lot of energy trying to make sure nobody else is impressed either.
Look, you don’t have to worship the thing. But pretending it's just a fancy parrot with a glitchy brain is getting old. It’s smart. It’s flawed. It’s changing everything. Deal with it.
4 replies →
There's a lot in here. I agree with a lot of it.
However, you've shifted the goal post from AGI to being useful in specific scenarios. I have no problem with that statement. It can write decent unit tests and even find hard-to-spot, trivial mistakes in code. But again, why can it do that? Because a version of that same mistake is in the enormous data set. It's a fantastic search engine!
Yet, it is not AGI.
You say it's just a fancy search engine. Great. You know what else is a fancy search engine? Your brain. You think you're coming up with original thoughts every time you open your mouth? No. You're regurgitating every book, every conversation, every screw-up you've ever witnessed. The brain is pattern matching with hormones. That’s it.
Now you say I'm moving the goalposts. No, I’m knocking down the imaginary ones. Because this whole AGI debate has turned into a religion. “Oh it’s not AGI unless it can feel sadness, do backflips, and write a symphony from scratch.” Get over yourself. We don’t even agree on what intelligence is. Half the country thinks astrology is real and you’re here demanding philosophical purity from a machine that can debug code, explain calculus, and speak five languages at once? What are we doing?
You admit it’s useful. You admit it catches subtle bugs, writes code, gives explanations. But then you throw your hands up and go, “Yeah, but that’s just memorization.” You mean like literally how humans learn everything? You think Einstein invented relativity in a vacuum? No. He stood on Newton, who stood on Galileo, who probably stood on a guy who thought the stars were angry gods. It’s all remixing. Intelligence isn’t starting from zero. It’s doing something new with what you’ve seen.
So what if the model’s drawing from a giant dataset? That’s not a bug. That’s the point. It’s not pulling one answer like a Google search. It’s constructing patterns, responding in context, and holding a conversation that feels coherent. If a human did that, we’d say they’re smart. But if a model does it, suddenly it’s “just autocomplete.”
You know who moves the goalposts? The people who can’t stand that this thing is creeping into their lane. So yeah, maybe it's not AGI in your perfectly polished textbook sense. But it's the first thing that makes the question real. And if you don’t see that, maybe you’re not arguing from logic. Maybe you’re just pissed.
1 reply →