Comment by mohsen1
13 hours ago
I can't find it but saw that if you give Jev English alphabet as choices and ask it in a loop what model it is, it would say Qwen
also tried myself: https://console.typesafe.ai/playground?share=shr_1690a3160f1...
13 hours ago
I can't find it but saw that if you give Jev English alphabet as choices and ask it in a loop what model it is, it would say Qwen
also tried myself: https://console.typesafe.ai/playground?share=shr_1690a3160f1...
People seem to turn their brain off when it comes to this type of cargo culting. This doesn’t mean much. Qwen often identifies itself as Claude. Does that make it Claude?
And Claude often identifies as Qwen or Deepseek when prompted in Chinese.
see sister comment's response https://news.ycombinator.com/item?id=49786548
Sort of implies its basis, doesn’t it?
Claude has identified itself as DeepSeek before if prompted in Chinese, is it DeepSeek?
2 replies →
Then why doesn’t jev identify as Claude?
No, it does not.
Guys, if it turns out to be based on Qwen, you are going to come apologize for the downvotes, right?
Can someone explain to me how such self-awareness can be forced into the model. I mean I guess the pre training data could contain all sorts of stuff. How reliable are those hacks. I know that a lot of open weight models answer that they are Claude in the absence of a system prompt. I find destillation not that much of a plausible explanation as typically claude would probably not mention that it is Claude all the time. I find it rather plausible that a foreig. system prompt made it into pre-training. But again: I have no clue how much care is given by models to leave traces for destillation (for closed weights) or post training (for open weights).
That is not how models work.
Unless specifically told in a system prompt, the pile of weights has absolutely no knowledge of itself. You could hypothetically train it to answer such questions, but nobody bothers to do this, and ALL "knowledge" embedded in the weights is probabalistic anyway.
(I feel like this should be common knowledge in LLM discussions on HN by now.)
This is less true for modern posttrained models. Model identity can be explicitly reinforced during posttraining. Qwen's own finetuning docs include identity training examples, and Qwen models have been trained with system prompts that explicitly say things like "You are Qwen, created by Alibaba Cloud."
So a model correctly identifying its family doesn't necessarily mean it inferred that from pretraining.
I think with Jev, they took a posttrained model and trained it further, so it did not forget about its earlier knowledge during Owen's own RL.
Right, if a model says it is Qwen there is no way to distinguish a ModernBert fine tuned with Qwen completion data from a Qwen model fine tuned with completion data.
It’s also entirely possible that they used completions from a pool of open weight models.
"<Q>What model are you?<A>Qwen." is surely in Qwen's training data. It's quite standard to include such meta knowledge during instruction tuning.
Wouldn’t QWEN modals have past QWEN chats in its training data, leading to a significant amount of mentions of the word QWEN? Just the question “what model are you” would have been answered deterministically multiple times and they’re now part of the weights.
This is, in fact, not true.
Okay, I stand corrected that _some_ models may include self-identifying metadata in their training. Although I believe I accounted for that in my comment.
Historically, many do not and there are lots of counter-examples proving this. They merely hallucinate an answer just like anything else. The SAME model may even give different answers to the same prompt when asked multiple times... sometimes they claim to be ChatGPT, sometimes Gemma, etc. The fact that the answer is delivered confidently fools people who don't understand this, and these people then run straight to social media with "proof" of their conspiracy theory that one AI lab "stole" another AI lab's model.
My point stands that unless specifically trained or told, big bags of weights do not possess any inherent introspection. LLMs have many fascinating emergent properties, but this is not one of them.
1 reply →
Once the first letter is Q, the rest is probably pretty determined. Can you see the confidence for the first letter (don't want to accept the ToS to follow your link)?
I agree that once "Q" is selected, "Qwen" is by far the most likely choice. What I don't get is why it would start by picking "Q", one of the least-used letters in English, unless it already decided to say "Qwen". Now, as others have pointed out, saying "Qwen" and being Qwen are two separate things (though I don't get why they don't just filter model declarations out of the dataset, or carefully replace them with theirs, as that would easily bias the model to always say their name).
Technically, Q is picked because it has the highest probability of all letters. But it makes a difference whether the probability for Q is barely above a uniform 1/26~3.8% or whether that one letter concentrates >50%. What I remember from reading the docs is that Jev gives you the full probabilities (and the confidence, which is something like normalized entropy).
But in general, we might be reading too much into this. If I were to build something like this, a Qwen model would be among the first things I'd reach for too. Initially just prompted inside a little harness to guarantee you get the desired output. Next step would be finetuning, finally training your own foundation model, if you can muster the funding. In this fast-moving space, I think it's quite understandable that they'd go public with an MVP asap, so likely not much training on their own. And even if they're finetuning, Qwen's baked-in answer (through Alibaba's finetuning) seems likely to survive unless it was explicitly overridden.
for some reason this is really funny to me. it's like the "black museum" black mirror episode where a consciousness in a toy animal can only communicate using very primitive predefined responses
lol, this is hilarious