Comment by pixl97
16 hours ago
>How can such an obscure point in latent space be optimal for every model??
While I don't have any evidence based answers here, this could be some interesting mathematical artifact of the set of human knowledge and platonic algorithms.
Kind of like how carcinization turns some animals into similar shapes and forms, maybe there are just weird algorithmic shapes that emerge from how we use language. Of course we'd need to see more of these in different languages other than English to see if they disappear in some languages or appear more often in others.
This is just another one of those things about LLMs that we have no idea why it happens, and there are so many others already. And when you look at the pace things are progressing it seems we're building up new questions like this far faster than we are getting answers for them.
> This is just another one of those things about LLMs that we have no idea why it happens, and there are so many others already. And when you look at the pace things are progressing it seems we're building up new questions like this far faster than we are getting answers for them.
I think that this is most likely due to RLHF, as there's only a much smaller number of vendors here, and the likelihood is that the workers of the vendors preferentially prefer some kinds of actions.
I saw something like this in a previous life, where we had a small pool of human raters in cheaper places, which, when used as training data biased the models in a bunch of unexpected ways.