Comment by hedgehog
1 hour ago
What would you build? Flash Next on 128GB works ok, 3.8 27B might technicially work on less but it's just too slow at least on current APU style chips. I'm curious about the non-NVIDIA 32GB discrete GPU options but I haven't tried yet.
Flash next only needs 64GB for core model inference. If you really wanted it.
Need 64GB vram, 900+ GB/s speed, and a lot of system ram (128GB). Seems feasible. Hmm.
Maybe older GPUs work.