Comment by lambdaone
1 month ago
Instead of having distinct tokens, you could have modifier vectors which would be added to other tokens. Think in terms of control, shift, meta etc.
1 month ago
Instead of having distinct tokens, you could have modifier vectors which would be added to other tokens. Think in terms of control, shift, meta etc.
Hey, Jasmine here (one of the authors) -- that's an interesting idea!
There's an interesting exploration of this here: https://www.lesswrong.com/posts/HEzNZ9gvgYwT3aZFS/role-embed....
Curious if you have additional thoughts, and thanks for reading!