Comment by aetherspawn

Ah yes but because it’s a MoE 48GB active model, then it’s possible that we might be able to run it locally in specialised setups such as 256GB unified memory.

Many MoE models (seem?) to only require enough memory to load the active expert.