Comment by vladgur
19 hours ago
At cost of power at $.45kwh here and dual 3090 setup easily approaching 1kw per hour at 120 tok/sec, latest Luna is cheaper per million tokens.
Your electrical costs are likely less than mine though
19 hours ago
At cost of power at $.45kwh here and dual 3090 setup easily approaching 1kw per hour at 120 tok/sec, latest Luna is cheaper per million tokens.
Your electrical costs are likely less than mine though
A dual 3090 system shouldn't go above 600W wall power while running inference, even less with some weights in RAM as you need to do with "only" 48GB VRAM and this model. Unless you are running water cooling, the cards are likely sitting at 200-250W power limit each anyway so they don't run too hot. This is considering a consumer tier CPU and mainboard with about 90W TDP, not a dual CPU or something like a threadripper that can easily push the power draw to 1kw if it runs the calculations on the RAM weights. My dual 3090 setup usually drew about 200W-300W average with LLM workloads according to my HA historical data.
I'm working off a 128gb Strix system on mostly hydro and it's worth it for me. Qwen 3.8 27B and Flash Next, Gemma 4 and MOE models. It Keeps my thinking and methods local while moving at a speed I can actually read and think about.
It can be annoying when you're running something long and realize there is an issue. But hey, I'm not wasting paid tokens. Reset and try again. I've probably thought of a better way by then too.
Yeah electric costs are nil on the microgrid but capacity is limited.