Comment by pcblues
8 hours ago
Sensible thought. I very much hope there is a glut of one-three year old ram and GPUs on the market in about one year when the AI market "cools" and the ear-marked components return to the market.
The banks that lent the AI industry the money are already trying to sell their debt.
Unless you meant buying an entire server (instead of laptop/desktop components), it won't work out the way you are describing. Prices may come down, but the components for the datacenters market aren't fungible with the components for laptop or desktop. You might not know what is being "earmarked" in this case?
Yes, the DRAM dies all come from the same wafer supply and fab capacity, and those limits are the cause of the current prices. However, once the memory OEMs have packaged DRAM dies into something like an LRDIMM or SOCAMM, the cake is baked. It's no longer usable in a laptop or desktop. No amount of X-year-old LRDIMMs (hypothetically) flooding the market will be useful for anyone's desktop or laptop. And then there's HBM, where the dies are directly on-package with the CPU or GPU.
Second-hand, revalidated server DRAM components may contribute somewhat to a price decrease, but those won't be the components you or I will be purchasing (unless you run a true server platform as a desktop, in which case, shine on you crazy diamond!).
The same is partly true for GPUs: there are PCIe versions, but most are OAM or SXM modules. You might be able to jury-rig an SXM module into a desktop? Adapter cards exist for at least some SXM versions, and you could figure out the cooling somehow? But it's probably more trouble than it's worth.
I have some amount of confidence that the sellers on AliExpress will figure out how to create cheap hardware that makes use out of all of this when it hits the secondary market.
Yes, but also no?
For standard desktop CPUs, the memory controller doesn't support the signaling required to communicate with an RDIMM. There's no clever AliExpress adapter that will magically give a component within your CPU capabilities that it simply doesn't have.
However, if you have a true workstation, you don't even need some adapter from AliExpress! Xeon 600, Threadripper 7000, Threadripper 9000 all support RDIMMs natively.
Unfortunately most of it will likely be HBM and datacenter GPUs and therefore useless for consumers.
Datacenter GPUs are still self contained and speak over pcie though, right? So its a form factor and cooling problem, not an insurmountable one.
Maybe not insurmountable in the "laws of physics" sense?
However, it may be so costly in cobbled-together parts, and in time to deal with unsupported drivers and/or VBIOS, that it is not worth it compared to using a proper server hardware with proper SXM sockets. Nvidia is also doing a lot of new, exciting things with networking that may make their SXM-based GPUs require even more just-so hardware support as time goes on.
Getting older SXM3 GPUs (i.e. for V100's, from 2017) working via PCIe adapters has been done reliably. However, here is someone who did that successfully, and spent a chunk of time last year trying to do the same for SMX5 (H100) and failing:
https://forums.servethehome.com/index.php?threads/sxm5-h100-...
We were all spoiled by the era from 2005-2020, when you could squint at a "server" configuration and see that it was expensive, high-binned, commodity hardware with some extra RAS and OOBM features. You could buy parts harvested from a retired server based on Xeon E5-1680v3 and drop them in your workstation. Or you could buy an entire single-socket Xeon E5 v3 server, plop in a GPU, and mostly use it as a workstation! As-is!
Now the serious servers have SOCAMM memory, 400Gbps networking, EDSFF SSDs, maybe are configured for CXL 3.0, et cetera. The hardware itself is so divergent that it isn't swap-in-plug-and-play with desktops, even for some high-end workstations.
1 reply →