Comment by edude03

So for a sense of scale, un-quantized 1 parameter == 1 byte of ram. 12Gb means you could have roughly 12b parameters, frontier models like Kimi k3 has 3 trillion parameters, so you'd need ~3tb of vram.