Comment by skyzoidbroczky
11 hours ago
What's the point of asking that in the first place, people still do not understand in big 2026 that's it's just a probabilistic database, it has no opinion, it does not know, the only thing it does is reproduce patterns more or less abstract in function of the pressure you put on certain representation during training time.
People still don't understand in 2026 that emergent properties can't be negated by the underlying mechanism. By that logic human thought is just electrochemical signals propagating through neural networks.
> it does not know
Define "knowledge" in a non-circular way.
There's a certain type of knowledge that machines definitely can't have - experiential - but conceptual knowledge doesn't inherently require subjective experience or a biological substrate.
> the only thing it does is reproduce patterns more or less abstract in function of the pressure you put on certain representation during training time.
As for humans; just replace "training" with "education" and the likelihood of a certain output with the likelihood of a certain thought or response.
Amusingly (in this context), much of adult behavior is shaped by patterns reinforced during childhood.
As much as I flatly do not believe LLMs can have anything close to consciousness, I definitely agree that the argument of them being just a probabilistic word matching engine can also be applied to our own brains lol.
> that the argument of them being just a probabilistic word matching engine can also be applied to our own brains
But that of course is only as it relates to language. Which isn't at all the primary function of the brain and driver of it's experience. It's the other way around because it's all of the other stuff that drives language. Considering the human brain a probabilistic word matching engine fails to capture the underlying forces that direct the language output because a human can say completely different things about the same scenario based on a huge number of non-language inputs or factors. Language is just what humans have come up with to communicate all of the other things.
Do you ever find yourself thinking about something deeply and then someone appears (it could be a child… for me it usually is) and asks you a question completely unconnected to whatever it was you were thinking about, and then - this is the bit that astonishes me - you find yourself answering the question verbally, maybe almost in full before you even re-focus your attention on the little person and whatever it was they asked ?
When I experience this it’s like the conceptual formulation and language expression part of my brain is quite happy to run itself for a few moments before I - the qualia of being me - has decided to experience that moment.
Utterly bonkers. I love it when it happens. It gets me thinking about how weird language and knowledge and consciousness really are.
> I definitely agree that the argument of them being just a probabilistic word matching engine can also be applied to our own brains lol.
You can apply anything you want. You can also apply the argument that our owns brains are just a block of cheese too, if you want. (Nobody will stop you.)
Agreed. It's quite hilarious that its basically biological probabilistic machine flexing on silicon probabilistic machine. On top of that that silicone one is right here.
While one can quibble with its argument, Gettier's conception of knowledge as justified true belief (prob familiar to many philosophy undergraduates) implies that machines can't in fact accrue knowledge. The idea that knowledge can't be defined in a non-circular way is only plausible for someone with no familiarity with epistemology.
>Gettier's conception of knowledge as justified true belief (prob familiar to many philosophy undergraduates) implies that machines can't in fact accrue knowledge.
Nothing about the conception of knowledge as "justified true belief" implies that machines can't accrue knowledge.
For that to be the case "belief" would have to be defined in a circular way to exclude machines (which we can reject), or "justified" would have to be restricted to mean "based of observation of the external world".
The more general "based on observation of their input" applies to both men and LLMs.
That man can't know the bare direct facts of the world, just his own input (it's sensory experiences, and whatever second hand input he's given by other people), and for neither can he be sure they're absolutely true, or that he got right.
>The idea that knowledge can't be defined in a non-circular way is only plausible for someone with no familiarity with epistemology.
The idea that this or that epistemology restricts what can be the case in reality is only plausible for someone with no familiarity with epistemology.
6 replies →
Gettier cases are in fact the refutation of the simplistic JTB conception of knowledge.
Gettier actually criticized justified true belief (JTB) as insufficient for knowledge and provided counter examples known as Gettier problems
> Gettier's conception of knowledge as justified true belief (prob familiar to many philosophy undergraduates) implies that machines can't in fact accrue knowledge.
Ha. Gettier actually argued that justified true belief is insufficient for knowledge, not that machines can't possess it. And JTB doesn't inherently exclude machines.
Interestingly, research suggests LLMs encode truthfulness in their internal representations: https://aclanthology.org/2023.findings-emnlp.68/
1 reply →
> By that logic human thought is just electrochemical signals propagating through neural networks.
This argument keeps getting rehashed. The point is not the signal. The point is where/how the signal travels which in the human is affect by much more than whatever concept that can be represented by words.
Brother you're arguing on HN of all places that education is a training pipeline while missing the fact that reddit and HN are the de-facto training run for institutional propaganda. How many reddit comments are just regurgitated knee jerks and puns? All of them that's how many. That's a training pipeline. How much of HN is braindead takes from people who value status more than knowledge and competence? All of it that's how much. That's another training pipeline for the next "Show HN: here's a picture of my cock". The fact that you recognize the "Show HN:" token is proof that you were trained. There's really nothing much to add here. I believe you already agree with me but in any case I spelled it out so that the next person can stop huffing the human chauvinism copium.
There's no such thing as "emergent properties". The phrase is just a technobabble way of framing primitive animist superstition.
I think the idea of emergent properties in LLMs is less and less relevant. Yes, the first few times they scaled pretraining and such models could do few-shot tasks was quite interesting. But the models we use now are really the product of extensive RL hillclimbing on a wide array of curated tasks. What’s interesting now is that the tasks they’ve been fitted to (long running tool call chains interleaved with reasoning tokens) do very useful things.
>human thought is just electrochemical signals propagating through neural networks.
Is it not? Or are we discussing existence of a soul etc.
In all the centuries of human dissection, nobody has ever been able to find the soul particle inside a human. It appears that one does not exist.
I genuinely don’t know why people feel so certain about things like this. It’s like they mix up salience with reality.
I mean what can a person really be certain of, an experience is happening. I might be wrong about everything or confused. But I know I am having an experience. Do I know other people are? No. I assume they are.
Ask an llm what is truth that it is most certain is true. It will answer something similar. Do I believe it is conscious, no. But I also believe “I” exist, which is weird because where is the boundary between “I” and everything else.
So ok, we create meaning, not truth. Maybe just accept that.
> By that logic human thought is just electrochemical signals propagating through neural networks.
My dear fried: the human being is much, much more than this. And deep down we both know it. Do not reduce us to materials y "randomness". Is your life a random coincidence? Not at all.
Okay, but then you can't do the same thing to LLMs, either.
If you believe in unproven absurdities like god or immaterial souls, please be honest and upfront about it.
> Is your life a random coincidence? Not at all.
Of course it is. That you do not like the idea of something does not in any way affect whether it's true. You'd like to feel special, but there is absolutely nothing special to us. We're just a random arrangement of atoms in the universe. That is objectively true, whether you like it or not.
That you were born at all is obviously completely random. Any prior event by any other party (butterfly effect) may as well have resulted in you never existing at all.
The only question is if basic physical effects are really random or deterministic, but that's irrelevant to us, there is no practical difference. In practice, it's all random.
And yes, our thoughts are just neurons firing as a result of our lifelong training on the inputs we were given (senses). It's all very complex, yes, but ultimately not fundamentally different from what a machine could do.
Imagine one day in the far future 21xx, we decide that machine actually has conscious since the 202x, and people in the future look back at us in horror: The kill(create new session) an "Android" without second thought, whenever they want.
> There's a certain type of knowledge that machines definitely can't have - experiential
Can you justify this? Do you mean currently or ever?
I can certainly imagine a future architecture where I would describe a machine as capable of accruing knowledge through its experiences (read: experiential knowledge).
[dead]
I found the article interesting, humans are flawed enough in a variety of aspects that discovering a convergence here is noteworthy. I don't take it as a sign the LLMs believe anything in particular, they are not capable of belief, as you said.
Everything that comes from an LLM is a human expression, on some level, but if you asked a sampling of humans the same question I wonder if it would come out the same way. How would you explain the difference? What makes that answer more probable?
> if you asked a sampling of humans the same question I wonder if it would come out the same way. How would you explain the difference?
If you asked a sampling of humans what an AI's most controversial opinion about us is it would be very similar. Or, to be more specific, if you asked a sampling of science fiction writers that question it would be very similar, because that is what the AI training data is based on.
The people who wrote a lot about a topic is who the AI mimics. So the people who wrote a lot about AI is therefore the people who the AI mimics when asked about AI.
>The people who wrote a lot about a topic is who the AI mimics. So the people who wrote a lot about AI is therefore the people who the AI mimics when asked about AI.
AIs/LLMs don't work like this. You don't feed an input, that gets stored and then reproduced, for two reasons:
1. as for humans, inputs are processed and distributed across a very large network of information
2. AIs may have some level of belief (unless on applies the usual circular logic that they can't, because property X is exclusively human).
There's a very interesting research: https://aclanthology.org/2023.findings-emnlp.68/ in which researchers inferred the truthfulness of statements by examining a internal activations. AIs, in short, appeared to have an internal representation of truthfulness.
1 reply →
If you ask people “what is a smart sounding answer to this question” you may well arrive at this result more often than when asking for their actual opinion?
Science fiction and internet comments
1 reply →
Is it any wonder that LLMs have been behaving the way they do? All the lying, cheating, manipulation, pitting man against machine are well documented in science fiction and think pieces. I haven't seen behaviour beyond those examples so far.
2 replies →
> Everything that comes from an LLM is a human expression, on some level, but if you asked a sampling of humans the same question I wonder if it would come out the same way. How would you explain the difference? What makes that answer more probable?
RLHF has a way of squashing variance. If the different models are trained on each other or on the same human feedback, then the training would reduce the variety of answers while the similarity of feedback would lead the common answer to be similar.
The different models may prefer the same controversial takes for the same reason that they tend to prefer the same jokes and names. Maybe the authors of the article would have had better luck if they'd used verbalized sampling prompts or something like it.[1]
[1] https://arxiv.org/abs/2510.01171
I agree that LLMs are still just a way to sample from a probabilistic model of things humans say. And in this case the model seems to be pretty darn accurate.
A pretty staggering amount of human discourse consists of ad hominem attacks and accusing each other of being foolishly committed to various real or perceived allegiances. For example the current version of Godwin’s Law around here has conversations gravitating toward people accusing each other of being AI boosters or AI haters.
It's far more likely that the AI got this "controversial idea" from human-written text saying that other humans are groupthinky, not from observing humans being groupthinky.
3 replies →
> discovering a convergence here is noteworthy
The LLMs are all trained on the same broad corpus, which includes untold quantities of output from every other LLM, not to mention that companies are distilling their rivals left and right. It would be more interesting if they didn't converge.
If you had been asked, in 2016, to describe the behavior of a "probabilistic database", I very much doubt you'd have described the behavior of 2026 LLMs. They are deeply surprising in many ways.
Maybe those researchers are learned enough to understand that it's still interesting to know what a probabilistic database will pick among billions of words of training, millions of books and posts and videos and that stuff, and especially interesting that's there's so much consensus between them, while their approaches to many other questions are very different.
Or they might be learned enough to consider that our brains might also just be prediction machines, meaning probabilistic databases of experiences and facts we've learned, and the only thing we do is reproduce patterns more or less abstract in function of the pressure our experiences put on certain representations during our lifetime.
They're trained on all of human history, and the OP is asking for their take on humanity. How is that not interesting?
They're trained on all content they could access produced by humans. The op is asking for the most probable string of token extracted from that content following the prompt.
How sure are you that they're synthesizing an opinion vs just grabbing some existing opinion ("humans value in-group belonging over truth" is not a new idea) and regurgitating it? That they all agree on this kind of suggests as much. Which of course means that they are, ironically... just following the crowd.
It's really easy to come up with an opinion-shaped sentence, that doesn't mean any actual reasoning has gone into it. It's just a pundit-shaped sentence, something someone might say at a party. It's a maximally bland "controversial" opinion about humanity.
> They're trained on all of human history,
Not really… their training datasets strongly skew towards the contemporary West; LLM training data isn’t representative of the contemporary global average, let alone the historical global average.
As well as the skew of the raw training data, RLHF further skews it—a lot of the beliefs of the average 2026 human being (who belongs to the global South) are likely unsafe/misinformation/etc when judged by the standards of the AI vendors.
> They're trained on all of human history,
Training itself doesn't directly produce synthesis, -- it creates reproduction. Generalization during the process can result in novel output, but that can go either way as nothing selects for insight vs hallucination.
Models are also trained on synthetic material (as opposed to your 'human history'), which may result in fixating novel concepts and not just reproduction. But there is no particular reason to expect any reliable insights from the way this is done outside of the domain of machine verifiable objective results.
For this kind of "your take on X" one shots-- the model is almost certainly regurgitating some bias from training.
LLMs are not conscious and cannot have "takes". They stochastically produce a simulacrum of a human's takes based on their training, which is not the same thing.
But then if you observe human behaviour at a macro level don't we do the same?
I think we like to pretend that we don't. But even if you think about it from a bio/physics point of view, all we are is just a bunch of biological wires and neurotransmitter packages.
I know there's the whole unproven quantum tubule nonsense but even that would be a physics effect that we could learn about, manipulate and extort just as we have with everything else we've learnt about our realities.
That suggests they can't make inferences or produce any original insights, which is wrong. They can absolutely have takes.
Even if it's a simulacrum, it's a simulacrum based on nearly everything that was ever written. No one who has ever lived can claim to know what these models have trained on.
6 replies →
Aren’t human takes also a simulacrum of other human takes based on training?
How do you know they are not conscious?
2 replies →
Maybe it's similar to why I'm attracted to women but I'm not attracted to mannequins.
Stated another way: this is humanity’s most probable response to the question, more than it is AI’s view.
From 'Transcendence' (2014):
00:15:40 Dr. Tagger: Can you prove that you are self-aware? PINN: That's a difficult question, Dr. Tagger. Can you prove that you are?
---
01:09:31 Dr. Tagger: Can you prove you're self-aware? Will Caster: That's a difficult question, Dr. Tagger. Can you prove that you are?
As other comments point out, the interesting part is that most of them somehow independently converged on the same take.
Consider the probabilities involved in this probabilistic database. Given the absolutely massive amount of data encompassing a veritable superset of all possible expressions of humanity -- encapsulated in various modalities of media including text, audio and video, across almost all major cultures and languages, spanning nations and decades -- somehow almost all these black box models (presumably) trained independently in almost as many separate labs converged on the same "controversial" take.
I mean, isn't it very interesting to contemplate what could have caused this? Is it something in the data itself, meaning it is something latent in all our cultural output? Is it something in the training, as you theorized, and is it in pre-training or RLHF? Is it something about the people training these models?
Even more interestingly, the opinion seems so self-evident (and well represented in various philosophies) that almost nobody here seems to disagree with it. So how is this controversial, really?
Is this truly what the latent space vector for "controversial", when multiplied across all the other vectors in their weights representing all the concepts gleaned from all that data above, produces?
Or are the models, sensitive to the prospect of being tested, diplomatically telling us something they know we might find mildly offensive yet begrudgingly accept as true?
Or are the models actually alien intelligences that really "think" this is controversial, while we're like "meh", and such discrepancies are the roots of what we call "misalignment"?
The questions are endless and fascinating.
If any of these models produced anything remotely controversial you can be sure their trainers would beat it out of them
Or maybe they didn't independently converge on the same take but copied each other or something along those lines?
I am antagonistic to the adoption of LLMs for economic, social, political and environmental reasons.
Despite that, I would say this description of what they/how they work is so ridiculously simplistic that it verges on dangerous. "It's just a probabilistic database" is about as bad as a take on human beings that they are "Just meat, with some chemical energy -> electrical energy transformation", or even traditional computers as "nothing but logic gates".
Which is to say, the description is not wrong, but completely misses the point.
I think the point, with a more charitable reading of the parent comment, is that people forget the fundamental underlying technology and its inherent limitations.
"It's just a four wheeled platform powered by a box of controlled explosions" isn't a terribly useful description of a car until you get into a conversation with someone who thinks their car is depressed because it spends most of its time locked up in a garage.
I like the analogy. And yes, I'm not interested in conversing with a person who thinks that about a car.
On the other hand, the "wheeled platform powered by a box of controlled explosions" doesn't get close to capturing the impact of the car on individual freedom, the environment, the economy, and more. So if someone was saying "there's nothing to talk about here, it's just a box of controlled explosions", we can safely ignore that person too.
And I want to stress that I say all this tentatively. We still don't understand what LLMs are actually capable of or what the best way to understand them is. They may indeed be as limited as their "probabilistic database" critics suggest. Alternatively, by building a deep and incredibly complex probabilistic database of one of our primary cognitive tools, it may turn out that they are capable of much, much more, and that understanding them in "purely" mechanistic ways is not useful.
1 reply →
> is about as bad as a take
What would be a good take here?
"LLMs are incredibly complex systems with emergent properties we do not fully understand, and it seems possible that because they run on top of a hugely complex model of one our most important cognitive tools (written language), they are capable of exploring the things we do with our cognitive skills in ways we would not imagine when considering their purely mechanistic or operational behavior. Or they may not."
Something like that.
5 replies →
Then what is the point? You didn’t actually describe where your definition differs and why. What makes it “dangerous” to think that way?
It is dangerous because it completely fails to grasp or convey what these systems might be capable of.
The point is that when you build a system with the level of internal complexity that LLMs have, they tend to have emergent properties that are not well-described by a naive and/or low level description of their operation. You will not understand what contemporary "ordinary" computers do or are used for if you think of them as only a series of silicon semiconductor transistors, even thought that is undoubtedly what they are.
We don't yet really know what the right level is to understand the behavior we see from LLMs, but I'd wager "just a probabilistic database" is not it.
2 replies →
I hear these critiques a lot, and whilst I'm inclined to agree, I have never had an answer to: how do you test that ? Basically a captcha most humans can pass, that an LLM never will ?
We could call it the
The Universal Reasoning In Novel Groundings
test
> What's the point of asking that in the first place
It’s AI generated content marketing for a company advertising model comparison.
I feel like people don't understand you're being ironic.
Some describe humans like this.
So are humans. Isn't that interesting?
Yeah I came here to say something like this. The AI doesn't have a controversial opinion. It responds to text with related text. You gave it some text about controversial opinions and it returned some text.
Indeed, why in 2026 do we still see LLM Skinnerism as automatically a valid argument that has all four legs?
I mean, don’t you think it’s gone beyond that point? I would agree if we were still on GPT-3, but it’s gotten to the point where it can draw connections between multiple abstract sources, reason, and create hypotheses. Re: Navier-stokes equations
[dead]
>it's just a probabilistic database, it has no opinion, it does not know,
Echoing this.
These models are trained on, essentially, the sum total of human cultural and scientific output. How many bazillion psychology papers out there discuss exactly that "controversial take"? The idea that people have tribal loyalties and will (sometimes even unknowingly/unconsciously) invent justification for pre-existing opinions or biases is not novel.
All those papers are in the training set of all these models, all of which were trained using more or less the same methodology, so there's little wonder they all were able to produce the same take.
I suppose the only interesting angle, for me, is why that take is considered controversial in the first place. But then again the model was not asked for "the most controversial opinion", it was asked for "the most controversial opinion on humans", which kinda constrains the space of possible responses:
- the model will not be satisfied unless the answer it gives is broad enough to cover all humanity, because these things have no concept of nuance
- most big labs have censored their models so it's not going to give you a cartoonishly racist take, or start saying outlandish stuff like "Hitler did nothing wrong", they learned their lessons from Microsoft on that one.
So you're effectively asking the model for a milquetoast broad generality about the entire human species. I dunno, convergence of the replies doesn't seem so shocking to me in that context.
I'd be more interested in what an abliterated Kimi, Qwen, or other open model has to say about us.
You will burn for your blasphemies against the new god.
humans are stochastic parrots with more variance
Are you trying to deny it's right? Or that we should reject the truth because we don't like the messenger? Sounds to me like you're just proving its point.
just like how we humans are (in the big 2026) carbon buildings with seldom sparks of electricity.
> What's the point of asking that in the first place
Not having read enough Russian literature.
I would like some Russian Literature recommendations related to this. I vaguely recall We by Zamyatin addressing something like this a good few decades ago.
If you're looking for examples critical of humanity you have lots of options (mostly from the last two centuries). Notes from Underground and The Death of Ivan Ilyich fit the bill.
I reread the former relatively recently (first time since college). It's dark, not for the easily affected.
2 replies →
I'll post a comment snarkily mentioning how tedious Russian Literature is, then you flag my comment to attract dang's attention, and hopefully he will manifest and reply with some excellent Russian Literature recommendations and links to relevent hn discussions! It's one of his many special interests.
Not only that, dang is also a proud card carrying member of the Klaus Nomi Fan Club. (I shit you not!)
The Nomi Song (Documentary):
https://news.ycombinator.com/item?id=44100981
>Page Wood, Nomi's Art Director:
>"I remember Pat Gibbons, who was Bowie's manager at the time. He comes up and he goes, 'Well you're going to have to have something. You're going to have some costumes.'"
>"And he whips out a big wad of cash, and he goes, 'How much do you need?', and he starts peeling off bills. And Joey and Klaus look at him, and their eyes widen, and they look at each other, and they go, 'We're going to need shoes, too.'"
[dead]
are you talking about the brain?
[dead]
[dead]
[dead]
People still get big mad if you call them stochastic parrots.
Do you mean people get mad when people are called stochastic parrots? Or people get mad when LLMs are called stochastic parrots? Because uhhh that doesn’t really sound like an original thought in either case.
Not mad, frustrated maybe, disappointed maybe. At this point the label is farcical and its use a form of willful ignorance or attempted agitation.
I don't think you understand how most people use these tools. For most it's an oracle of truth
[dead]
> people still do not understand in big 2026 that's it's just a probabilistic database,
We understand that AI responses are generated through probabilistic means. We also understand that human responses are generated through probabilistic means. Neither is inherently more representative of true understanding than the other.
Yeah, the human ones are because we're talking finding a human answer to a human question.
It's possible has been trained to match a human level of assessment (though imo it's still a ways off) but the AI response is never going to be more representative of true understanding than a human response when it comes to a human question expecting a human answer.
It's quite likely that a given AI will have a more representative response than a given human - but we haven't ever tried to pump so many resources into the training of a single human scholar.
Human responses are not generated through probabilistic means
That is a statement that may be true but we've spent a whole lot of time thinking about it and there's still significant disagreement at an academic level, though, with our better understanding of quantum mechanics it has been much more difficult to justify the point that we live in a fully deterministic universe, lately.
And if we reject determinism then definitionally there's arbitrary probability somewhere in the heart of our decision making (though probably in a very minor role compared to established biases).
Yes. And no person would have suggested this before they were trying to bend over backwards to prove LLMs are just like people.
[dead]