Comment by romirain2007

Well, guess what, transformer is also a "traditional" SVM that assigns a 0-1 label: https://openreview.net/forum?id=U_T8-5hClV

It is interesting that you have cited this paper but did not even correctly acknowledge their contribution. Yeah I get all that "they are doing X and we are doing X+1" narrative, but the fact that you have defined "good" tokens by multiplying Y_i to your head function, is not much different than "assigning 0-1" label to inputs in traditional SVM. Your "Y_i" essentially serves as a 0-1 label in SVM.

Sounds like a mind game of re-branding existing concepts lol.