Comment by wafngar

Should be a relevant reference: https://arxiv.org/abs/2111.13587

Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers John Guibas, Morteza Mardani, Zongyi Li, Andrew Tao, Anima Anandkumar, Bryan Catanzaro