Comment by naasking

19 days ago

The linked paper tested nanoGPT with this new transformer:

https://www.techrxiv.org/users/685780/articles/1375955-topol...

3 comments

naasking

Reply
tuned  19 days ago

thanks for linking.

Yes the paper compares the new architecture (that is also a fork of my implementation of nanoGPT) with Karpathy's nanoGPT. There are also links to the code and bench used.

  • Herring  19 days ago

    Note I didn't say Karpathy's nanoGPT, I said use the speedrun.

    Transformers are universal function approximators. When well-tuned, they often start to approximate other innovations. Not always, thank god, but often enough that you have to be careful.