Comment by lovich
3 months ago
I ran the 32b parameter model just fine on my rig an hour ago with a 4090 and 64gig of ram. It’s high end for the consumer scene but still solidly within consumer prices
3 months ago
I ran the 32b parameter model just fine on my rig an hour ago with a 4090 and 64gig of ram. It’s high end for the consumer scene but still solidly within consumer prices
I'm confused. According to other comment: https://news.ycombinator.com/item?id=42859645, <= 70b DeepSeek models are just a fine tuning of Llama or Qwen? So we shouldn't take any thought of these models to actually being DeepSeek.
I think people are confusing the smaller non-DeepSeek original models (Qwen/Llama) with the 700B DeepSeek R1 model being talked about in here and that very few people can run locally.
I run the 32b parameter model also just fine on our 4x H100 rig :) It's good enough for embedding, our use-case.
I'm not sure if $200k of hardware fits the consumer level
I have also been running the 32b version on my 24GB RTX 3090.
if someone wants to run the real thing (R1) locally, someone posted their hardware specs on X. Total cost: $6,000.
[0] direct link with login https://x.com/carrigmat/status/1884244369907278106
[1] alt link without login https://threadreaderapp.com/thread/1884244369907278106.html
That's not DeepSeek, it's a Qwen or Llama model distilled from DeepSeek. Not the same thing at all.
I am doing the same.