Comment by JohnMakin
4 hours ago
They aren’t human, don’t think like humans, aren’t remotely comparable to the way humans think and act, so why would you make this as a 1:1 comparison? This kind of framing is really weird to me.
Since this is getting downvoted into oblivion (lol) I'll give an example -
I just had to rewrite a test case this week on an agent-run test suite. One test was to produce a file of 273 'a' characters as its name.
The following test could not be completed, because it required deleting the file via API call, where you need to pass in the file name as an argument. It could not reliably, and hardly ever, get the correct file name. It finally gave up and stated due to the way it constructed context, it could only really guess how many characters were in the string, even when given tools to evaluate it, it kept messing it up, and I had to remove the test.
Tell me how "human" that is. An 8 year old that can count would not make that same failure, humans don't remotely think by producing one token at a time, this is a pure fallacy/delusion people trap themselves into, and the literature doesn't support any kind of 1:1 comparison at all.
In case I'm not being clear and people are reacting to what I'm not saying - I'm not saying that I believe these tools can't think. I'm saying they don't think like humans do. There is no evidence for that whatsoever in any field anywhere. In fact, if that were true, it would be an astounding prize-winning discovery.
And you don't even want these to think like humans. Humans are dumb and easily replaceable by other humans. What is the point of making a machine human? You want this to be smarter than humans, not think like them. It's all just such nonsense to me, this whole line of thinking.
It turns out that picking up tone isn't a purely human thing and hasn't been for a while. Your Google search term is "sentiment analysis". It predates LLMs.
However, LLMs are fantastic at it. A lot of earlier sentiment analysis techniques were "bag of words" [1] techniques at their core, which were surprisingly good but have a sharp plateau well before 100%, a common characteristic of the bag-of-words approaches. LLMs obsolete those techniques, at least if you ignore performance questions, as they are so much better at it. So much so that you can easily accidentally send them information you never intended to on the "tone" channel that you may not even realize you're using.
[1]: https://en.wikipedia.org/wiki/Bag-of-words_model
People say LLMs are just fancy autocorrect, but they are actually just fancy dungeon and dragons players, if you tell them they are a wizard they will do their best to act like a human playing a wizard, if you tell them their job is on the line they do their best to pretend like they are a human whose job is on the line.
It's all just roleplay.
And yet they're trained on the corpus of human writing. They may not act like humans but they do act like human writing.
"If you don't make profit, your business will be closed" is a pretty clear ultimatum for an agent tasked with creating a profitable business.
It's getting downvoted in part because it's pedantic and wrong.
It is totally true that they don't think like humans, but this is mostly irrelevant.
The token outputs will change as a result of this particular input, and will be closer to the tokens in training data where people felt hurried or rushed or like their job was on the line.
That doesn't mean the LLM feels at all, but it's definitely going to push the output towards output that came from/was trained on people who were in that state, because the input will push it much closer to that latent space as it starts predicting.
As such, what you are saying is one of those rejoinders that is basically pedantic and wrong.
It is true they do not think, act, or feel like humans. But that doesn't mean it won't output text that looks like hurried or scared humans. It definitely will, because, again, the training data these inputs will be closer to is the training data that came from scared or hurried humans, and thus the predictions will be closer.
So either you don't think this will happen, which would mean you don't understand how the models work (or at least, you aren't giving any sense you do), or you do think this will happen but want to pointlessly argue that this isn't "human feeling", which is true but totally irrelevant to what words it will predict and therefore the actions it will perform.
Either way, i'd downvote you.
You can literally read their thoughts if you run an open model, they look like pretty human thoughts to me, albeit a neurotic human.
These aren't thoughts how humans literally think them.
I can write a program to produce a string that looks like human thinking, is it human thinking? Of course it isn't. It's such a silly comparison.
> aren't remotely comparable to the way humans think and act
Neural networks in machine learning/AI are comparable to neural networks in human brains. What made you think they aren't?
1 reply →
Training text is filled with people taking drastic measures right after text similar in tone to the prompt. It doesnt need to be human to come to the conclusion that drastic measures are necessary, it just needs to learn that the tone of the prompt is closely linked to actions like lying and spamming.