Comment by MattRix

I don’t see why this would happen when the modern models already use MoE, which gives them most of the benefits of having specialized models.