Comment by rektlessness
17 hours ago
Deepseek released just enough to stir the pot but not enough to cook your own from scratch.
Wake me up when we can clone, point it at a dozen H100s we own, and get a working production service.
Anthropic and OpenAI will do just fine.
> point it at a dozen H100s we own
uhm...wake me up when I can buy one(1, single, uno) GPU* to upgrade my PC so that I can play modern games with high graphics, without feeling like I'm being bled dry.
y'all got dozens of H100 at home? isn't that ~300k??
*actually my bigger problem is that I need a new CPU, which means I need a new motherboard, which means I need new RAM, which...you get it.
This is not a home/consumer GPU discussion.
But in that vein, you are in the same position as the chinese LLM providers. They cannot get hardware to host so they put the problem in the forefront by offering it open.
In a way, Deepseek is championing your cause.
> This is not a home/consumer GPU discussion.
fair point...
> In a way, Deepseek is championing your cause.
maybe in 2030 i'll have a chinese gaming PC with chinese local AI. would be interesting to see if they can do the same to US tech companies as they do to european car manufacturers. i'm ready for it...
You can ask Claude to finish it
Make no mistakes.
Target is by end of Q4.
Anybody could fire up vLLM now to do exactly what you're talking about. And they do? There's dozens of places hosting DS4.1 Flash, at good speeds. So I'm not sure what your point is.
What this provides is the give contributors to vLLM, sglang, etc. a reference implementation of some of DeepSeek's own hard-won inference serving techniques.
No doubt we'll see patches to vLLM inspired by bits in here.