Comment by roenxi
13 hours ago
Ram prices suggest there is going to be a mighty glut of the stuff in a couple of years. We may easily see 32->64->128 GB system RAM off the back of that and CPUs seem to be moving to unifying VRAM and main memory.
I don't think running a ~200B parameter model is that out of reach, it might be usual by the mid 2030s for people in the tech world. Factor in the crazy improvements in quality given a number of parameters too.
But why? HBM is not reusable by consumers, and we would probably be at ddr6 by the time used ddr5 comes crashing down.