Comment by albertzeyer
13 hours ago
The random noise is added to the model parameters, not the inputs, or not?
This reminds me of variational noise (https://www.cs.toronto.edu/~graves/nips_2011.pdf).
If it is random noise on the input, it would be like many of the SSL methods, e.g. DINO (https://arxiv.org/abs/2104.14294), right?
Yes you are right it's applied to the parameters, but other models (like ngcm) applied it to the inputs. IMO it shouldn't make a huge difference main point is you max differences between models.