Comment by nerdsniper
5 months ago
It looks like this requires 1.5TB of VRAM? Did I get that wrong? What would be the least unreasonable way you host this without quantizing it?
5 months ago
It looks like this requires 1.5TB of VRAM? Did I get that wrong? What would be the least unreasonable way you host this without quantizing it?
3 m5 mac ultras with 512 each? idk