Comment by _ache_
4 days ago
Even a 5090 can handle that. You have to use multiple GPUs.
So the only option will be [klein] on a single GPU... maybe? Since we don't have much information.
4 days ago
Even a 5090 can handle that. You have to use multiple GPUs.
So the only option will be [klein] on a single GPU... maybe? Since we don't have much information.
As far as I know, no open-weights image gen tech supports multi-GPU workflows except in the trivial sense that you can generate two images in parallel. The model either fits into the VRAM of a single card or it doesn’t. A 5ish-bit quantization of a 32Gw model would be usable by owners of 24GB cards, and very likely someone will create one.
> Even a 5090 can handle that. You have to use multiple GPUs.
It takes about 40GB with the fp8 version fully loaded, but ComfyUI can (at reduced speed), with enough system RAM available, partially load models in VRAM during inference and swap at need (the NVidia page linked in the BFL announcement specifically highlights NVidia working with ComfyUI to improve this existing capacity specifically to enable Flux.2) to run on systems with too little VRAM to fully load the model.