Comment by rockinghigh

The MoE experts are quantized to int4, all other weights like the shared expert weights are excluded from quantization and use bf16.