Comment by krzyk

People are using 3090 (24GB) to run models, and it is the most cost effective way to run the. Yes, it is 2x faster, but memory wise you surely can spend 24gb on llm.

Also there are smaller, still usefull models that can run on 8GB or less.