Comment by virgildotcodes

Am I misunderstanding or are the models also limited to those that can be run with less than 96 gigs of VRAM?

The models that are both open source and quantized so that they can fit within that much memory are going to be significantly less capable than full scale frontier closed source models, I wonder how the latter would perform.