Comment by 29athrowaway

I big kudos to the author of this.

Not only has the visualization, but it's interactive, has explanations for each item, has excellent performance and is open source: https://github.com/bbycroft/llm-viz/blob/main/src/llm

Another interesting visualization related thing: https://github.com/shap/shap