Comment by muyuu

for unified memory, the dense models are way too slow and for local GPU-based setups, large MoE are too large but they're fine on unified memory systems

essentially, hardware is the main reason you may choose one or the other locally

i have a Strix Halo system so I will be trying this Dwarf Star 4 thingie eventually when i have some free time