Comment by lelandbatey

The dense is better in its result, but is out of reach for most hardware compared to the MoE models, and the MoE models produce output tokens much much faster.