Comment by adamhsn

Cool project!!

It would be useful to filter which model to use based on the objective or usage (i.e., for data extraction vs. coding).

Also, just looking at VRAM kind of misses that a lot of CPU memory can be shared with the GPU via layer offloading. I think there is ultimately a need for a native client, like a CPU/GPU benchmark, to figure out how the model will actually perform more precisely.