Comment by dekhn
1 day ago
I can't imagine spending $48K on a home GPU server, but I did just splurge and buy a PC with an RTX 5090, specifically to hold the largest models you can fit in 32GB. It's a top of the line PC with water cooled high end CPUs, 64GB RAM, RTX 5090 for $5K. To me the jury is still out whether this was a worthwhile investment, but I do expect to use this machine for a decade. I don't run it at 100% power (it's mostly idle, except for times when I'm training or doing batch inference). It has the nice property of being blackwell generation, similar to the machines we use at work.
It just scares me to own a box that is $48K in my house, especially if it breaks, or gets stolen.
Last fall, seeing the writing on the wall, I pieced together an "AI" rig, 96GB ram, 2x RTX 3090, 9950X - not exactly top of the line, but it came in around CDN$3000 all in all, with most parts second hand. I don't think I could build that for CDN$10000 today.
I've been using it pretty steadily for a variety of personal projects, and the only improvement (aside from the obvious "more VRAM") I feel pressed to make is a portable AC unit / some kind of a focused cooling solution. The rig raises the ambient temperature in the office by 4C at least.
Now with the murmurs of even the large players reconsidering their AI spend, and usage-based pricing shifts, having a self-contained, owned, and independently administered compute resource is looking better and better.
Well a lot of people have that in their garage, even "worst" it's on wheels so even easier to steal.
I'm not saying it's worth it just that it's not such a crazy amount in comparison.
>> To me the jury is still out whether this was a worthwhile investment, but I do expect to use this machine for a decade.
The high cost and power consumption are both signs of the death of Moore's law, so you are probably correct that this system will be near state of the art for some time.
I was looking at Ultras for sale, and had same worry, so didn't end up getting one. I have some peace of mind comfort about applecare and technical repair, but i couldn't find insurance that would cover theft (or rather, i did, but it was too expensive)
Yes! It scared me too. I tried to insure it under my renter's insurance policy, but they not surprisingly refused. I had to get business insurance to cover it
You showed this setup to a business insurance underwriter and they gave you a policy? Can I ask how much the premium is? Or is this just theft insurance?
Not even a single mention of gaming.
No wonder gamers hate AI bros.
I have a second computer with an RTX 4090 for gaming (running Windows). I also used the new RTX 5090 running Linux to evaluate whether Proton/Wine allow me to run Windows games on linux (yes, it works, but the compatibility and frame rate issues make me stick to native Windows for now).
If you want a GPU that has comparable performance on Linux to Windows- you want AMD. NVIDIA drivers are notoriously bad. Many of my games run better on Linux with the open source AMd drivers. (CachyOS rolling rolling rolling).
5 replies →
https://www.protondb.com/
I wonder what's going wrong there? Personally I found compatibility and performance on Linux to be extremely good. And just keeps getting better. And that's not even just me, that's all kinds of benchmarks out there. Sorry to hear that. : ' (
1 reply →
I have the same rig as you minus watercooling, and I assume you have AMD Ryzen 7 9800X3D? Anyway, it's my only PC now, I game, dev, run local models, edit photos, edit videos, all in Manjaro. I get ~70FPS in Cyberpunk at 4k, every setting at "Insane" or whatever goofy thing they call it, Ray tracing on path tracing off, with no framegen but with DLSS set to quality. Without DLSS I get around 40fps. Seems equivalent to what I see online with people with a similar build on Windows.
I run hyprland, seems to be the only wayland based keyboard-forward WM that has good nvidia support (and, allegedly, supports HDR, though I haven't got this working). I heard gnome was pretty good otherwise. I was running i3 before and it also worked fine, however once I got into wanting to get streaming working, there wasn't good compatibility between i3/xorg and tools like sunshine. I believe steam streaming worked fine on it though iirc.
The only thing I miss from windows: easy streaming with sunshine/moonlight. Steam streaming works (usually heh) but it took me a couple days of fiddling to get a stream to work at all through sunshine, and it is choppy. But for local gaming, I don't miss windows at all, I'm so glad to finally have all my drives converted from NTFS to ext4.
1 reply →
Having built an almost identical rig earlier this year can promise at least one similarly-spec'd machine gets equal use between AI and gaming (Both on Linux). Stupid-excited for the Steam Frame to finally come out.
or crypto... what's old is new again.
I would probably hate someone if they were buying the same hardware as me but doing something actually useful with it. Any game worth playing doesn't require high specs anyway. There is such a large catalog of old games.
I specifically got the previous model so I could play AAA games with all the settings set to Ultra, at 4K. Cyberpunk 2077 struggled even with my 4090, so I had to disable ray tracing and enable DLSS. Since I've run out of new AAA games I've been playing older ones and it's crazy how fast they are.
I don't think you can dismiss gaming as not "actually useful".
> No wonder gamers hate AI bros.
Personally, playing with AI models is way more fun than getting sucked into a game loop. Game loops feel like busy work hooked to an engineered dopamine drip. AI models are new frontiers and are exciting to build with, modify, lobotomize, and hack around with.
And some of us are doing AI stuff all freaking day at day job and just want to play some Tekken when we get home for 30 minutes after the kid is in bed. But now Playstations are 1000$ and Ram and GPU prices are astronomical.
Not everyone is hustling 24/7 like some kind of lunatic.
1 reply →
I remember playing Quake III which had user-programmable bots and thinking "wow, this is a really hard computer vision and reasoning problem". And then realizing "huh, that's a major research area, I should work on that". Later I learned that the bots were fairly simple and worked on far simpler world representations (nav meshes).
It looks like DM took a crack at it: https://deepmind.google/blog/capture-the-flag-the-emergence-...