Comment by zozbot234

Running a 247GB model reliably on 100GB VRAM total is a very impressive outcome no matter what the performance. That size of model is one where sensible people will recommend at least 4x the VRAM amount compared to what you were testing with - at that point, the total bandwidth to your storage becomes the bottleneck. Try running models that are just slightly bigger than the amount of VRAM you're using and these tricks become quite essential, for a significantly more manageable hit on performance.