Comment by rurban
5 months ago
I run the 32b parameter model also just fine on our 4x H100 rig :) It's good enough for embedding, our use-case.
5 months ago
I run the 32b parameter model also just fine on our 4x H100 rig :) It's good enough for embedding, our use-case.
I'm not sure if $200k of hardware fits the consumer level