Comment by XCSme

Yes, specific weights/parameters have be trained to solve specific tasks (trained on different data).

Or did I misunderstand the concept of MoE, and it's not about having specific parts of the model (parameters) do better on specific input contexts?