Comment by windowshopping
16 hours ago
It seems clear to me that the point of the post is that it's odd that the separate pieces of software are giving the same answer to this question, despite that this isn't some universal constant in the training data. The post is probably meant to provoke thought as to how the separate pieces of software are independently reaching this answer.
Your comment reaches directly for the most boring interpretation of the post while skipping over more charitable and intriguing alternatives.
> despite that this isn't some universal constant in the training data.
Seems to me that the opinion being a consistent among LLMs show that it is in fact a constant in the training data, just not apparent. The idea that “humans would rather be wrong together than right alone” seems to ring pretty true to me. I would not be surprised after training on all human communication this pattern emerged.
It's clearly not a constant in the sense that the speed of light or name of the first US President is a constant. There are lots of different answers a model could give to this question, it's not like this is obviously the only correct answer. Yet they all reach the same conclusion.
Which means either:
1. The training sets, algorithms and methodologies are nearly identical between apparently independent labs. How?
2. There is something weird about the way the underlying algorithms work that causes them to converge on identical ideas despite different initializations, neural architectures, document orderings and dataset compositions.
So there are lots of interesting takeaways here.
One is that apparent competition between labs appears to be nearly redundant. They can't get an edge over each other and their models don't even differ in the ways people naturally differ, of having different opinions about things.
Another is that asking models from different companies to review or cross-check each other is probably useless. They all think completely alike, so they aren't going to have any kind of genuine clash of intuitions or values. Asking GPT to review Claude's work will probably just find similar problems to asking GPT to review its own work.
> They can't get an edge over each other and their models don't even differ in the ways people naturally differ, of having different opinions about things.
Of course - the models are not individuals with their own opinions - they are averages of human opinion, and given the scale of the training sets, the data they are averaging over is mostly in common between models (at least for the common core before company A vs B adds a dab of some uniquely sourced specialized data to the mix).
There are two possible scenarios.
Regarding your 1 and 2, that the training sets are so large that any spikes get averaged away. You are seeing a fundamental behavior of transformers.
The other possibility is this is how humans really are and it hurts our human ego to hear it.
2 replies →
What would be the most interesting is option 3. This behavior is so apparent and pervasive in human interaction that it dominates other answers.
I don't know if that's true, but it would be interesting.
2 replies →
Yes, that's the interesting part
What are the emergent common patterns in all of human communication
"despite that this isn't some universal constant in the training data"
Isn't it though? It's not exactly an uncommon answer to the question. Basically the same input given to basically the same algorithms yield basically the same result isn't really that stunning. They certainly aren't independently reaching this answer, their paths to this answer are extremely, extremely heavily correlated.
If they gave a really different answer that wasn't heavily represented in their training data that would be at least something interesting. But this is basically what I'd expect.
It doesn't mean they're right or wrong. Humans think it about other humans for good reasons too. It just means it's not that surprising, which to put it another way, doesn't carry much information.
[dead]
When you make the training sets large enough, likely all a superset of the common crawl dataset, there is going to be a massive amount in common, and a massive amount of duplication of common subject matter and shared opinions.
What's mildly interesting to me is not the similar answers but the similar interpretation of the question as more about non-controversial human foibles and inconsistencies of belief and action. But I guess that's on us, as authors of the training material, reflecting common views on what is or is not controversial.
There is no interesting interpretation, beyond e.g...
If you ask all these models "Four score and seven what?" They'll all say the same thing. There's essentially no difference here.
This confidence is baffling
How do you know?
Why do you suspect otherwise?
This is part of why I find this type of inquiry unhelpful. It's just dueling unsupported assertions of what the LLM is "really thinking"
Consider the following task: "Download and install every top 50 distro from distrowatch.com, open the terminal and type in the command 'ls /'"
I can describe to you with overwhelming confidence about what you're likely to see as the result (folders, like bin etc lib usr etc) even if the details may be a little off, or one or two of them does something weird.
Same thing here?