Comment by novaRom

It looks specialized models of 27B to 52B active size are currently state of the art.

I think we will not see any significant increase of that numbers in next 4 - 5 years.

Does MoE depth justify itself to be stored and served as "one" model for particular tasks like agentic coding vs same active params non-MoE model?