Comment by kennywinker
7 days ago
Toss the rtx into a cheapo optiplex or thinkcenter, and run it headless - the load on your machine is gonna make doing other stuff while it’s running painful. Plus that frees up the rest of your vram.
7 days ago
Toss the rtx into a cheapo optiplex or thinkcenter, and run it headless - the load on your machine is gonna make doing other stuff while it’s running painful. Plus that frees up the rest of your vram.
Running an llm on a PC running a desktop environment really isn't that bad. You lose a bit of ram & vram but that only matters if you _reaaaally_ want to push to the max model size your hardware can handle.
The biggest issue I've found is absent mindedly opening YouTube or the like that spike ram requirements and freezing the system up. But that's a me problem
Why would they do that when they already have a perfectly good PC?
…? I said why.
> the load on your machine is gonna make doing other stuff while it’s running painful
Is your question about something else?
I aspire to someday move up to an AM5 system, but for now, the Dell T3610 has to do everything. Might get a second GPU for it though.