Comment by teleforce
19 hours ago
Someone has modified microgpt to build a tiny GPT that generates Korean first names, and created a web page that visualizes the entire process [1].
Users can interactively explore the microgpt pipeline end to end, from tokenization until inference.
[1] English GPT lab:
I have no affiliation with the website, but the website is pretty neat if you are learning LLM internals. It explains: Tokenization, Embedding, Attention, Loss & Gradient, Training, Inference and comparison to "Real GPT"
Pretty nifty. Even if you are not interested in the Korean language
This kind of thing is pretty easy to do with a much leaner model https://docs.pytorch.org/tutorials/intermediate/char_rnn_gen...
By "modified" this person of course means that they swapped out the list of X0,000 names from English to Korean names. That is seemingly the only change.
The attached website is a fully ai-generated "visualization" based on the original blog post with little added.