Comment by DonaldFisk
4 hours ago
It's telling that king – man + woman = queen is the only example I've ever seen used to explain word2vec.
I prefer the old school
king(X) :- monarch(X), male(X).
queen(X) :- monarch(X), female(X).
queen(X) :- wife(Y, X), king(Y).
monarch(elizabeth).
female(elizabeth).
wife(philip, elizabeth).
monarch(charles).
male(charles).
wife(charles, camilla).
?- queen(camilla).
true.
?- king(charles).
true.
?- king(philip).
false.
where definitions are human readable rules and words are symbols.
The difference is that Word2Vec "learned" these relationships auto-magically from the patterns in the surrounding words in the context in which they appear in written text. Don't forget that this was a revolutionary result at the time, and the actual techniques involved were novel. Word2Vec is the foundation of modern LLMs in many ways.
I've seen in readings (and replicated myself on a set of embeddings derived from google books/news) the capital cities:
Berlin - Germany + France = Paris , that sort of thing
What’s Hamburg - Germany + France?
In Qwen3-Embedding-0.6B that's Marseilles
And Munich - Germany + France is Strasbourg
again,
is clearer.
Someone once told me you need humongous vectors to encode nuance, but people are good at things computers are bad at, and vice-versa. I don't want nuance from computers any more than I want instant, precise floating point calculations from people.
I think you are missing the difference between a program derived from training data and logic explicitly created. Go ahead and proceed to continue doing what you are doing for all words in the dictionary and see how the implementation goes.
1 reply →