Comment by lukewarm707

I like semantic space because it is all continuous. As I spend more time using embeddings I sometimes think in semantic space.

A father is continuous opposed to boolean. 'Father' would rank highly in semantic similarity to a man who has a child, but father-like is better, easily someone is more or less father-like. A woman without a child has a certain value of 'father'.

But really the direction 'father-like' is not an intrinsic feature of semantic space, it is a human construction. Much the same as it would be arbitrary to measure 'v shape'.

I don't think that the latent space has a property or point 'v shape'. My view is that there are no concepts or neurons there. Discrete abstractions are not intrinsic properties of the model. Actually I think the abstractions of concepts or neurons do not map onto anything.

A nice example from cot itself is pareidolia. That is, seeing faces in clouds, or jesus on a piece of toast.