Comment by giancarlostoro
10 hours ago
> This is pretty awesome. The only compute I have at home is an RTX 3080 with 10 GB of VRAM, so I struggle with training larger models (>40M, 50M params). I get OOM errors and have to optimize a lot.
I'm on the same GPU, its intimidating to me if I even want to bother training anything at all. Do you mind sharing what kind of training you've done with that GPU? :)
Make sure you are running adaptive cooling (or just bump them up) on the top fans in your case. Also, ensure that you are undervolting appropriately using something like MSI Afterburner or GreenWithEnvy.
If you don't, you could easily toast your RAM -- especially under BF16.