Comment by nl

> Small and fast experts with a router LLM on top to best use them

A router LLM isn't a MoE.

A MoE is a type of LLM architecture, not lots of different LLMs. They are fundamentally different concepts and it is a fundamental misunderstanding to conflate the two.