Comment by Lucasoato
3 hours ago
My question is: what kind of hardware do you need to run this Flash beast locally at a meaningful speed?
3 hours ago
My question is: what kind of hardware do you need to run this Flash beast locally at a meaningful speed?
8x RTX PRO 6000 or 4x Spark? Or 1x M5 Ultra 512GB.
The model is theoretically FP8, but really internally its mostly FP4 already, so there won't be a cut-in-half-but-almost-just-as-good quant coming for this one.
a beefy pc with at least 20 GPUs