Comment by estimator7292
2 days ago
Sure, I think it's pretty interesting that given the same(ish) unthinkably vast amount of input data and (more or less) random starting weights, you converge on similar results with different models.
The result is not interesting, of course. But I do find it a little fascinating when multiple chaotic paths converge to the same result.
These models clearly "think" and behave in different ways, and have different mechanisms under the hood. That they converge tells us something, though I'm not qualified (or interested) to speculate on what that might be.
Two things that narrow the “unthinkably vast input data”: 1) You’re already in the latent space for “AI representing itself to humans”, which has a far smaller and more self-similar dataset than the entire training corpus.
2) We’re then filtering and guiding the responses through stuff like the system prompt and RLHF to get a desirable output.
An LLM wouldn’t be useful (but might be funny) if it portrayed itself as a high school dropout or snippy Portal AI.
Instead, we say “You’re GPT/Gemini/Claude, a helpful, friendly AI assistant”, and so we end up nudging it near to these concepts of comprehensive knowledge, non-aggressiveness, etc.
It’s like an amplified, AI version of that bouba/kiki effect in psychology.