At the beginning of William Gibson’s Neuromancer, the protagonist is trying to sell 3 MB of RAM in underground markets. This is often cited as one of the ways the book has not aged well. But, looking at the direction of the memory market now… maybe we just haven’t gotten there yet.
Early computer scientists were so optimistic. They beleives with a few kh of ram and a mhz of cpu they could do anything. Ai, consciousness, ml, language, text to speech. Now we spend gigs of ram on web forms. So gibson saying yeay 3MB of ram would probably be enough for a consciousness in cyber space, is very optimistic but fitting.
Not sure if "optimistic" is the right word - you could still do a lot with tiny memory or CPU footprint, but that's difficult to do if a large part of tech have adopted to either not care about the waste ("space is cheap"/ "the RAM would just sit there unused if I didn't use it") or lately even based technologies on the paradigm of using as much of it as possible. That was the explicit idea of bitcoin, but even AI development goes by the logic of "what would happen if we just made the model twice as large?"
The last iteration is "tokenmaxxing" where you try to spend as many tokens as possible first and then find out if it got you anything useful.
"Early computer scientists were so optimistic. They beleives with a few kh of ram and a mhz of cpu they could do anything." -- this isn't true, much less the stuff layered on top (conciousness!?)
In early computing history, the unit of memory was not established yet, and different hardware architectures had different word sizes, not even necessarily evently divisible by 8. And the memory sizes of these machines used to be expressed using the naitive word size. Like “this machine has 8 kilowords of core”. Therefore, when I encounter an anachronistic memory size in old fiction, I just assume that I just don’t know the word size they are using.
Gibson was also not a microcomputer enthusiast at the time he wrote the book, and didn't know much about the inner workings of PCs that ordinary people could afford with a regular amount of money in the late 70s/early 80s. As I recall he said he wrote the book on a typewriter. He's also on record as saying that the first half of the sequel in the series (Count Zero) was also written on a typewriter.
edit: But regarding your specific comment, it's true that memory size wasn't totally consistent in how it was described or marketed. But in terms of computers that you could purchase and fit in your house, if you look at things similar to Byte magazine from 1982, 1984 or whatever, it was pretty consistent that desktop microcomputers were specified in multiples of KB of RAM like a 64KB or 128KB commodore (or at the very low end 16KB or 32KB for something you would attach to your TV), or IBM PC/Clone that would ship with 512KB or 640KB or 1MB of RAM, and the more expensive ones having multiples of 1MB, 2MB, etc.
Star Trek used the fictional "gigaquad" unit of computer data storage, and conveniently deliberately avoided defining how many bits the base unit of a "quad" is, so they never would get embarrassed if such an amount of storage became commonplace nowadays.
Memory in particular is something that I've reflected on more than once as having the most impressive gains in computing since I started paying attention to it (networking/USB too, but that doesn't make your computer "faster" in the same way).
I remember being able to borrow a computer from somewhere when Diablo II had just come out in 2000 which had a 450Mhz Pentium III and 64 MB of RAM. 64MB of RAM was probably mid-tier at the time, i.e. very much not a given. As I recall Diablo II recommended 64MB for single player and 128MB for multiplayer (or above 4 players or something).
The computer I'm writing this on has 64 GB of RAM, 1024 times as much. By comparison I have a 20-core Intel CPU with up-to 3GHz speed or somewhere around there, even pretending each core could run at that max speed simultaneously (which they can't), that's only 133-times as much CPU power.
Maybe the NVMe read times are as/more significant than memory size increase, but the metrics on them isn't quite as front and center on PC specs as memory and CPU.
Hard drive capacity similarly impressive as RAM in terms of size (was apparently 10-30GB in 2000), but I don't have a 10TB hard disk as I don't need one that big (1TB is plenty for me), so again it's not as impactful to me as memory.
> The computer I'm writing this on has 64 GB of RAM, 1024 times as much. By comparison I have a 20-core Intel CPU with up-to 3GHz speed or somewhere around there, even pretending each core could run at that max speed simultaneously (which they can't), that's only 133-times as much CPU power.
Over that time CPUs have also increased their instructions per clock by 3 to 4 times, so the comparison is a bit closer than that. 5Ghz in CPUs is also common these days which would make it even closer. RAM has also improved in more than just total size though.
> The computer I'm writing this on has 64 GB of RAM, 1024 times as much. By comparison I have a 20-core Intel CPU with up-to 3GHz speed or somewhere around there, even pretending each core could run at that max speed simultaneously (which they can't), that's only 133-times as much CPU power.
This nerd sniped me a bit. Your calculation on the amount of CPU power is too low, because of the change in IPC, but for the things we have benchmarks for, it isn't multiple orders of magnitude off like I expected. Looking at Cinebench 2003, prime95, and a few other benches, I get somewhere between 300x and 850x faster for the modern CPU over the Pentium 3.
For me, the biggest change in performance in my life was going from spinning disks to SSDs. That change felt bigger than any other leap by a long shot.
> Maybe the NVMe read times are as/more significant than memory size increase
This was the most impactful upgrade/breakthrough for me. The first time I put even a SATA SSD in my PC at home I was completely blown away. It still blows my mind somewhat the amount of compute I have sitting on my desk though, both in terms of memory and CPU/GPU power, but that move from spinning rust to solid state was huge.
Then Apple did to me again with the M1 launch and NVMe speeds that made swapping nearly imperceptible.
> Diablo II had just come out in 2000 which had a 450Mhz Pentium III and 64 MB of RAM. 64MB of RAM was probably mid-tier at the time
I like to use Google books to refer to old issues of PC Magazine.
For $1999 in Feburary 1999, you could get the Pentium 450MHz desktop with 128GB of memory.
That said, I could do almost everything I do today on a similar machine back then. Surf the web, admin Linux servers, web development, edit video, play games, Photoshop, IRC, type papers...
akshually, it's also more closer to 500-1,000x. You can't look at clock speed only. Processor architecture makes all the difference. Pipelining, SIMD, memory bandwidth, blablala, everything got way better. Better approximation would be to use something like a synthetic benchmark or just (theoretical) FLOPS of each.
That is a pretty nitpicking reason to say it has not aged well. Hamlet doesn’t have cell phones yet I think it’s an excellent play. Even though a quick FaceTime would’ve averted a tragedy.
Here’s another fun nitpick from Neuromancer. The opening line says the sky was the color of a TV tuned to a dead channel, which in Gibson’s day would have been analog static that looks like a messy grey from across the room.
But later, with computerized channel tuning, a dead channel was shown as a screen of solid bright blue, and even later, solid black. So different generations of kids have grown up with very different mental images of the background lighting for the opening of Neuromancer.
Hamlet is set in (what is currently) the past. It is self consistent. Neuromancer is set partially in the future, and partially what is our past. The inconsistency is what throws people. It can be a good book, but people might still find those elements jarring.
So many plots in Seinfeld would have been trivially solved with cellphones. Get separated in the parking garage? Call each other. Need help carrying an armoire? Call each other. Trying to meet up at the movie theater? Call each other.
Side note. It is sort of the unwritten rule in TV and Film. Don't highlight technology unless absolutely necessary. it can age it considerably.
A few weeks back I had '30 Rock' on, it was funny seeing everyone with flip phones, it wasn't highlighted but it just makes you realise the age of it all.
I always just find stuff like that charming. I love seeing relics of the time in which something was made. I love seeing dated pop culture references, it's one of my favorite things about engaging with old media.
Can't you just read "3 MB of RAM" as a large amount of some scarce tech resource and move on?
What if you got a on-chip compression algorithm so advanced that you can fit a world in a few MB and now with corporations controlling memory distribution, 3MB of high compression memory is highly valuable in the black market.
"High compression memory" is immersion-breaking magic. There are hard mathematical limits to how much you can compress things (Kolmogorov complexity), and every lossless compression algorithm necessarily increases the size of some inputs (pigeonhole principle). It's much simpler to assume that "M" is slang for some bigger unit.
We already have the real life example of people using "mega" (10^6) as slang for "mebi" (2^20).
It's funny... I enjoyed Neuromancer, although I didn't read it until about 15 years ago.
And, yeah, the memory thing hasn't aged well. Thing is, 1984 was a funny time in computing, particularly when you consider the kind of computers normal people had access to.
At that point even things like PCs and the new Mac had 128 or 256K of RAM[0], so I get that 3MB must have seemed like an ocean of memory at the time. And, realistically, more than 1MB of RAM in machines you'd typically see sat at home or on a desktop was uncommon until the beginning of the 1990s.
And, although Moore's law had been around since 1965 it's hard to know how aware people outside of specialist circles would have been of it in 1984.
I suppose Gibson must have done some pretty in depth research for Neuromancer, right? But the memory thing is sort of ancillary to the story, so how much would he really have focussed on that? Probably not much.
And then do you really want to harshly judge the book on that one slightly laughable thing, in other ways, it was incredibly forward looking and almost prophetic? Doesn't seem right.
I think the sensible position is you have to let it slide and see it as a possible alternative future that never quite came to pass in that way but that which we can see strong echoes and foreshadowings of even still.
[0] In 1984 microcomputers, as opposed to, cough, "serious" computers like the PC and Mac, with 128K of RAM were still very new, with 32 - 64K being the entry level, and if you had one with 128K you were king of the hill. 128K in 1984 seemed like a ton of memory to most of us, but it's worth bearing in mind that only a handful of years before computers like the ZX81, which had only 1K of RAM, were the common entry level, so the progression was already clear if you looked at the situation in the right way, but you had to have been paying attention for a while to have noticed. I remember the first time I used a machine with 4MB of RAM in, maybe, 1990 - an Archimedes at school - and feeling like it was just this absolutely inexhaustible ocean of memory. In 1984 3MB would have felt almost inconceivably huge unless you were in the high performance computing, or maybe the mainframe, worlds.
> But then Gibson must have done some pretty in depth research for Neuromancer, right?
Isn't he on record that his documentation was listening to techies talking shop in bars?
> And then do you really want to harshly judge the book on that one slightly laughable thing about memory when, in other ways, it was incredibly forward looking and almost prophetic.
He seems to understand humans. Gibson's world and Brunner's Stand on Zanzibar are imo the most "prophetic" sf books written so far.
I've always wanted to set up automation that updates the text for the book every so often to preserve original intent, by changing the language around quantities to <whatever would make a contemporary reader be impressed>
Replace "megabyte" with "exabyte" or whatever.
As an aside, there's a great essay I think in Metamagical Themas about the fuzzy task of translating literary works, which takes up the question of whether it would be valid and how to translate say Dickens into French, by relocating from London to Paris.
Part of the premise is that this is an impossible task because the referential systems are not truly analogous; the cultures aren't identical so some concepts literally have no direct translation...
...in Neuromancer the mere existence of a bank of pay phones has stood out as one of those things which even my little updater would break hard.
Also related: I've seen a slowly increasing number of complaints in book forums about text from relatively recent novels being silently updated, to change references from e.g. Myspace to Tik Tok or whatever, to try to keep the text feeling current... a perilous slope.
And the bank of pay phones is one of the coolest little scenes in the book.
Put me on team “don’t update the text.”
Books are written in their contemporary cultural contexts. LeGuin said speculative fiction is about today even if the story is set in the future, and I agree.
So when you read Neuromancer, in some ways you’re reading a book that is about the 1980s in the U.S. So there are more fundamental anachronisms than just RAM and payphones, like a belief that east Asia had a lock on the future of advanced technology. Or that punk culture was cool and edgy and would endure.
From a quick skim, you could think of this as roughly equivalent to shoving a large amount of DDR4 on a PCIe card and using it as a swap space. It's more sophisticated (see CXL protocol), but that gives you an idea of the tradeoffs. It seems there is some OS-level support for moving hot/cold pages between the main fast DRAM and the expansion higher latency DRAM.
It's a very valid point that DRAM has a fairly long lifetime and contains significant embedded carbon emissions, as well as the current availability crisis of new DRAM.
That's just El Reg being El Reg (the Zuck-buck rhyme was apparently too good to pass up). But it's a far cry from their glory days, when they coined nicknames like "The Beast of Redmond" for Microsoft or "Chipzilla" for Intel...
I have always wondered why there was never a big market[1] for "cheap PCI/PCI-X/PCI-e card you can stick a boatload of your old/surplus/n-generation old simms/dimms on and use as swap/slow memory/ram disk/etc". It's rare you can populate a motherboard with a full address space full of 'new' memory, and you can teach kernels to prefer some memory to others because of speed[2], so it seems like a no-brainer.
I seem to remember the market for doing similar with flash got neutered over patent issues, but I can't recall the details. And flash cache did end up being a market, at least for bigger players. Maybe something similar happened here, or maybe it just hit a niche I cared about at the time?
[1] I know there were a handful of products in this space, but my impression is they never really took off. I could be wrong.
[2] Definitely can in NetBSD; I've done it for archs like VMEbus where it's common to have a small, fast on board memory and much slower, often larger memory out on the bus. I assume this sort of thing is enabled in Linux by the work to support NUMA, but I've never looked into it.
There was never a big market for it because new memory was not prohibitively expensive in comparison to the cost, risk, and limitations of using old memory in a new server. That is not the case now, so people are looking at the idea again.
I would counter tho that 1) this isn't the first time there's been a memory price/supply crunch, and "I've got a drawer full of last gen memory I can't use" is kinduva IT cliche, and 2) 'more memory' has always been a pain point, especially with industry practices like chipsets only supporting relatively small physical memory relative to address space (e.g. all those Intel LGA775 chipsets that capped at 4 or 8GB). Oh, and 2a) 'faster disk' has always been a pain point...
But, yeah...obviously my impression of things doesn't match market reality.
Not exactly the same but I'm building a NAS box with old parts. Most of my spare RAM sticks are laptop DDR4 SODIMMs. There are SODIMM -> desktop DIMM adapters... it did not go well. The system would boot 1 out of 5 times. No adjustment of memory speed settings would make the system stable.
That's not surprising. At the speed DDR4 runs, PCB design has to account for all sorts of weird effects. Adding the additional traces on the adapter probably pushes all sorts of timing over acceptable thresholds. There's a reason one of the big textbooks in the area is subtitled "A Handbook of Black Magic". [1]
Old generation DIMMs are not that much cheaper and supply is limited as old generation are taken out of production to free capacity for newer chips. DDR2 is already more expensive than DDR3, likely because it is no longer produced but there is still demand (to replace/upgrade memory in older hardware).
My understanding (which could easily be wrong), is the big difference today is CXL which adds cache coherency on top of pci-e.
Without cache coherency, you have to be more careful about how you use the memory and the performance story is complex. Ram over CXL is going to have worse perf than ram on the cpu memory controller, but there shouldn't be any big gotchas.
You could give it a similar structure to CPU cache or a giant swap file. Have a giant blob of memory on the PCI RAM, that feeds the on board smaller motherboard RAM as needed. It might just work.
Nvme drives already max out the 4x pcie lanes they get. You'd basically need to use the GPU slot to do better and even then you could do it with SSDs. M.2 break out cards are pretty common.
However unless we're taking 3DXpoint which was discontinued flash wears out. But yeah sticking an old optane nvme on a breakout card accomplishes much the same thing.
Cost for me. I wanted a Gigabyte I-ram but it was too expensive when I only had one 512MB DDR stick after upgrading to 4GB of DDR2. I bought a 60GB Sandforce SSD that fulfilled that speed gap.
I have always wondered why there was never a big market[1] for "cheap PCI/PCI-X/PCI-e card you can stick a boatload of your old/surplus/n-generation old simms/dimms on and use as swap/slow memory/ram disk/etc"
Reminds me of the days of JBOD arrays. Mac OS X had built-in support for it.
Consumer hardware costs are external. We are in the era of ignoring externalized costs, for example, Windows 11 hardware requirements, social media harm, and mainstreaming of gambling. Maybe the pendulum will swing back in the future, but today, line-going-up is the prime directive.
Necessity is the mother of invention, after all. (One of the oldest abstract concepts in intellectual thought, I suspect.)
There is a tight resource starvation/motivation loop — the demand put on RAM and SSD and GPUs by the largest frontier models is a direct motivation to make smaller LLMs. Like an evolutionary pressure making animals smaller and more food-efficient.
These smaller models, once successful, are still likely to consume more RAM and SSD and GPUs than any other application short of high quality video processing itself (the smaller LLMs and higher end video processing seem to have about the same needs). But the resources would distribute through the market more traditionally, leading to less insane cycles.
So it seems to me that the way out of the RAM/SSD price cycle crisis that manufacturers are in — where the price fluctuates between high and low due to supply constraints and then oversupply from new production capacity - is for them to fund research into smaller LLMs. They'll still sell essentially the same amount of product. Maybe more.
CXL Vistara reminds me a bit of the AST Rampage 286 memory expansion ISA card I had in my 286 back in the day, as a kid. Things go in circles, I guess.
observation that I've noticed recently: what's with wikipedia downsizing the hell out of images site wide? Every image I look at is garbage and I have to dig through multiple links to find the original.
Adding the RAM to the system this way isn’t exactly like expanding the main system RAM. The RAM is connected over a PCIe type link so it doesn’t behave like the primary RAM.
It’s better for server farms where engineers can customize and tune for an architecture like this.
There have been some cards that use RAM as a storage device. They were never popular because having to set it up as a disk had very limited use cases.
To someone not familiar with CXL that still gives the wrong impression. As far as I have seen, CXL is supposed to be cache coherent, and should require less invasive rework (if any at all) of applications to take advantage of it; that's part of the enablement of memory disaggregation that CXL is pushing towards (similar to the storage disaggregation push a decade or so ago).
The sensibility is a bit different since consumer systems don't really have much bandwidth back to the CPU. Given the current resale prices of DDR4, might as well just sell it and get some NVMe drives.
It will be interesting to see what happens to the consumer electronics market the next few years. Companies are right now gambling that consumers will pay extra because of RAM shortages. I suspect with the cost of everything else rising as well, a large portion of consumers (remember, HN, not everyone makes tech money) will just not be buying new devices for a bit.
I think I'm in this boat. I'm just choosing to interact with technology less. Everything has just gotten more hostile that it has reached a tipping point.
I'm with you on this one. I'll be using my 11th gen i5 and GTX3080 for the foreseeable future. The 'hostility' you mention puts me off doing so more than price of an upgrade.
I wonder if it would ever start to make sense to burn an AI model into ROM, replacing a large portion of an inference machine's RAM with ROM. (Probably not, since I'm sure those machines do dual-duty and run training when the inference workload slows down)
That's the idea behind Taalas (https://taalas.com), except as silicon rather than ROM. They run a demo at https://chatjimmy.ai/ which serves an old open weights model (Llama 3.1 8B) at something like 15,000 tokens per second.
Companies are building chips specialized for inference, so dual use for training isn't necessarily a consideration, but there are other considerations such as:
Weights need to be loaded into the accelerator's processor fast, which means they need to be physically adjacent to it, but there is limited physical space for that - not enough to fit the all the weights of a 1T+ param model, so weights get loaded into VRAM dynamically according to what part of the model is being run.
ROM (I guess we're talking Flash memory) can be dense, since it is built vertically - many hundreds of layers, but this comes at the cost of poor performance, so even if you could fit enough ROM next to the processor it would not be fast enough.
> In the future, hardware is only for big companies to own. At least it seems we're heading that way.
China is desperate to sell anything to... everyone. If there's a market, they'll eventually be there to fill it.
It took them decades for cars, but now they did it.
For RAM, CXMT went from 20 000 wafers per month to... 240 000 wafers per month in something like two years. And they're extending capacity massively now. It's a company only 10 years old.
The market is there and China shall flood it: that's how they operate with everything.
At some point they'll probably even come with GPUs that shall do 80% of the job for 20% of the price.
Just like you can buy chinese server motherboards at 1/5th the price of a SuperMicro one today.
So I'm not sure hardware is going to be only for big companies: China is going to put pressure on the OpenAI and Anthropic of this world locking all the RAM / SSDs / chips of this world.
> At some point they'll probably even come with GPUs that ...
I'd already like to play with the huawei card but everything is so overpriced at present and I don't think they sell those on the consumer market to begin with.
I think there's a great deal of underestimation of China's manufacturing. Granted, wafers are a totally different thing than any of the other industries they've dominated for cheap, but I certainly wouldn't count them out.
I've recently gotten into fountain pens. Sure, a $7 Jinhao or $15 Hongdian pen isn't going to write quite as nice as a $200+ pen, but they're about 80% of the way there, and you can buy tons of them for the cost of a single more expensive pen. Plus, some models will accept Western nibs just fine which means you're buying a cheap barrel and assembling a much higher quality product for almost pennies on the dollar.
One would do well not to underestimate their ability to fill markets. It may take years, but it will happen.
With regards to RAM price I never understood the following: A 16GB RAM stick has 16*8=128 billion bits, with 1 transistor per bit, thats still 128B, yet its supposed to cost like $60 before the price hikes? In contrast, a 5090 GPU was $2000 (true it has RAM, but you're paying for the GPU ASIC really, I guess the rest of the GPU was less than $500), it had 93B transistors.
GPU transistors are smaller due to the more advanced process node (cost per transistor metrics aren't really clear, if they improve on advanced node or not, but I'd say they get cheaper as they get smaller, as technology costs are amortized).
I'm sure both RAM and logic use a process that is quite similar in both inputs and manufacturing steps. So while RAM is a commodity product, this insane price difference didn't make any sense.
So I guess when those fundamental inputs become a constraint, it would make sense for $/transistor move closer for both, which is a massive hike for RAM.
Chip fabrication processes are not fungible: GPUs and CPUs might be made on roughly the same process, but DRAM is not (flash is a different process again, as is power electronics, analog electronics, MEMS, etc. And even within those broader categories there are different variations). While there are some overlaps in machines and techniques, a fab set up for one cannot generally switch to the other, and the economics of each process can also be drastically different.
You're not the first person to say so (and I don't mean to dispute it), but I have never been able to find a clear answer for /why/ those processes are incompatible.
Is it built in different silicon, is it physical steps that's incompatible (ie its actually incompatible), is it different physical preparations that needs to be made (making it economically infeasible to combine)
I cannot help but wonder, even if the answer doesn't change anything in my life.
RAM is literally copy-paste of the same circuit over and over, you're trying to compare cost to produce million AK-47 with 1 carrier
and cost per transistor stopped decreasing at ~20-30nm, now small nodes are targetting energy efficiency (and thus performance, since heat is the main limiter)
On top of everything said, 5090 die size is 10x than typical DDR5 die size. One RAM module is 8-16 dies, so you do get more silicon in the end, but larger dies are extremely expensive to produce due to sharply decreasing yields.
The thing that defines performance of DRAM is AFAIK the capacitor of the bit cells and not the transistor driving it. And also AFAIK the process to create those capacitors is quite unique to DRAM, so you can't just go and use a "logic" process unchanged and produce DRAMs.
newer process node are smaller but very expensive compared to mature ones, each wafer from TSMC latest process is costly and with lower yield due to GPU large die size (+700mm2 compared to around 60mm2 per DRAM die)
RAM is a commodity. It has much less moat to prevent competitions. When the rams flood the market that's when the bubble ends, until the next cycle arrives. Processors are much harder to design and commoditize.
Because you are paying for silicon, and processes, not transistors. Wafers have a certain cost, and litographic processes can illuminate a certain X mm2 of dies in an hour. If a transistor gets smaller, more of them fit in a certain area.
Granted the machines that make them become more expensive, but that's capital expenditure, which gets amortized as time goes on.
So there are two forces here working against each other.
> So while RAM is a commodity product, this insane price difference didn't make any sense.
Supply and demand coupled with the fact that a RAM fab can't (trivially) output compute chips, and vice versa, a compute fab can't output RAM. It's two completely different supply chains.
A GPU Transistor is a lot more complicated than a RAM transistor and the size of these are quite different too. Bleeding edge vs. a known process with know machines and written off machines.
Also you calculate in the machine cost and R&D.
RAM hiked because the demand spiked and these companies are now in power. Before apple and other companies told them the prices and had hardly any money for investment.
I am have an old Pentium 4 with RDRAM, think I could sell it to them? I think it has like 256MB. Haven't turned it on in awhile. Hope the first 640KB still work.
It'd be interested to see how one could leverage all the DDR3 ECC that they may have laying about; maybe an overseas shopping site has these boards available? Would DDR3 be as fast or faster than an SSD?
I fail to see why you couldn't also literally just stack RAM chips themselves on top of each other. If it works for stupid consoles (see NES/Dreamcast/PS1/PS2 hacking of recent), PCs should be no different.
At the beginning of William Gibson’s Neuromancer, the protagonist is trying to sell 3 MB of RAM in underground markets. This is often cited as one of the ways the book has not aged well. But, looking at the direction of the memory market now… maybe we just haven’t gotten there yet.
Early computer scientists were so optimistic. They beleives with a few kh of ram and a mhz of cpu they could do anything. Ai, consciousness, ml, language, text to speech. Now we spend gigs of ram on web forms. So gibson saying yeay 3MB of ram would probably be enough for a consciousness in cyber space, is very optimistic but fitting.
Not sure if "optimistic" is the right word - you could still do a lot with tiny memory or CPU footprint, but that's difficult to do if a large part of tech have adopted to either not care about the waste ("space is cheap"/ "the RAM would just sit there unused if I didn't use it") or lately even based technologies on the paradigm of using as much of it as possible. That was the explicit idea of bitcoin, but even AI development goes by the logic of "what would happen if we just made the model twice as large?"
The last iteration is "tokenmaxxing" where you try to spend as many tokens as possible first and then find out if it got you anything useful.
1 reply →
3MB of RAM but 120PB of storage. Sure you're paging a lot but
1 reply →
I remember when Johnny Mnuemonic came out and he was hauling 320 GB in his brain and that was a WHOA moment.
2 replies →
Well it was before electron was created so all they knew were efficient native applications
I like to remember that Gibson wrote Neuromancer on a typewriter and hadn't even touched a computer till (I believe) half way through Count Zero.
I saw a chrome tab this week that had Gmail with an empty inbox idling at 2.8Gb. Hard refreshed the page. Still 2.8Gb.
1 reply →
Maybe there is a parable here: don't fear the man that wants thousands of gigabytes, fear the man that only wnats 3 MB.
"Early computer scientists were so optimistic. They beleives with a few kh of ram and a mhz of cpu they could do anything." -- this isn't true, much less the stuff layered on top (conciousness!?)
In early computing history, the unit of memory was not established yet, and different hardware architectures had different word sizes, not even necessarily evently divisible by 8. And the memory sizes of these machines used to be expressed using the naitive word size. Like “this machine has 8 kilowords of core”. Therefore, when I encounter an anachronistic memory size in old fiction, I just assume that I just don’t know the word size they are using.
Gibson was also not a microcomputer enthusiast at the time he wrote the book, and didn't know much about the inner workings of PCs that ordinary people could afford with a regular amount of money in the late 70s/early 80s. As I recall he said he wrote the book on a typewriter. He's also on record as saying that the first half of the sequel in the series (Count Zero) was also written on a typewriter.
edit: But regarding your specific comment, it's true that memory size wasn't totally consistent in how it was described or marketed. But in terms of computers that you could purchase and fit in your house, if you look at things similar to Byte magazine from 1982, 1984 or whatever, it was pretty consistent that desktop microcomputers were specified in multiples of KB of RAM like a 64KB or 128KB commodore (or at the very low end 16KB or 32KB for something you would attach to your TV), or IBM PC/Clone that would ship with 512KB or 640KB or 1MB of RAM, and the more expensive ones having multiples of 1MB, 2MB, etc.
The ALU of a computer with a megabit word must be a beast to awe upon!
Star Trek used the fictional "gigaquad" unit of computer data storage, and conveniently deliberately avoided defining how many bits the base unit of a "quad" is, so they never would get embarrassed if such an amount of storage became commonplace nowadays.
Memory in particular is something that I've reflected on more than once as having the most impressive gains in computing since I started paying attention to it (networking/USB too, but that doesn't make your computer "faster" in the same way).
I remember being able to borrow a computer from somewhere when Diablo II had just come out in 2000 which had a 450Mhz Pentium III and 64 MB of RAM. 64MB of RAM was probably mid-tier at the time, i.e. very much not a given. As I recall Diablo II recommended 64MB for single player and 128MB for multiplayer (or above 4 players or something).
The computer I'm writing this on has 64 GB of RAM, 1024 times as much. By comparison I have a 20-core Intel CPU with up-to 3GHz speed or somewhere around there, even pretending each core could run at that max speed simultaneously (which they can't), that's only 133-times as much CPU power.
Maybe the NVMe read times are as/more significant than memory size increase, but the metrics on them isn't quite as front and center on PC specs as memory and CPU.
Hard drive capacity similarly impressive as RAM in terms of size (was apparently 10-30GB in 2000), but I don't have a 10TB hard disk as I don't need one that big (1TB is plenty for me), so again it's not as impactful to me as memory.
> The computer I'm writing this on has 64 GB of RAM, 1024 times as much. By comparison I have a 20-core Intel CPU with up-to 3GHz speed or somewhere around there, even pretending each core could run at that max speed simultaneously (which they can't), that's only 133-times as much CPU power.
Over that time CPUs have also increased their instructions per clock by 3 to 4 times, so the comparison is a bit closer than that. 5Ghz in CPUs is also common these days which would make it even closer. RAM has also improved in more than just total size though.
3 replies →
> The computer I'm writing this on has 64 GB of RAM, 1024 times as much. By comparison I have a 20-core Intel CPU with up-to 3GHz speed or somewhere around there, even pretending each core could run at that max speed simultaneously (which they can't), that's only 133-times as much CPU power.
This nerd sniped me a bit. Your calculation on the amount of CPU power is too low, because of the change in IPC, but for the things we have benchmarks for, it isn't multiple orders of magnitude off like I expected. Looking at Cinebench 2003, prime95, and a few other benches, I get somewhere between 300x and 850x faster for the modern CPU over the Pentium 3.
For me, the biggest change in performance in my life was going from spinning disks to SSDs. That change felt bigger than any other leap by a long shot.
1 reply →
> Maybe the NVMe read times are as/more significant than memory size increase
This was the most impactful upgrade/breakthrough for me. The first time I put even a SATA SSD in my PC at home I was completely blown away. It still blows my mind somewhat the amount of compute I have sitting on my desk though, both in terms of memory and CPU/GPU power, but that move from spinning rust to solid state was huge.
Then Apple did to me again with the M1 launch and NVMe speeds that made swapping nearly imperceptible.
2 replies →
> Diablo II had just come out in 2000 which had a 450Mhz Pentium III and 64 MB of RAM. 64MB of RAM was probably mid-tier at the time
I like to use Google books to refer to old issues of PC Magazine.
For $1999 in Feburary 1999, you could get the Pentium 450MHz desktop with 128GB of memory.
That said, I could do almost everything I do today on a similar machine back then. Surf the web, admin Linux servers, web development, edit video, play games, Photoshop, IRC, type papers...
https://books.google.com/books?id=mi_RGvUW6eQC&pg=PA108-IA3&...
2 replies →
that's only 133-times as much CPU power
akshually, it's also more closer to 500-1,000x. You can't look at clock speed only. Processor architecture makes all the difference. Pipelining, SIMD, memory bandwidth, blablala, everything got way better. Better approximation would be to use something like a synthetic benchmark or just (theoretical) FLOPS of each.
Otherwise, we can say that 6502 at 15Ghz is better than what you have now: https://news.ycombinator.com/item?id=22859706
The funny part is that a 1000x increase in RAM somehow doesn't make a modern computer feel 1000x more luxurious
2 replies →
3MB of "hot" RAM, stored in a cryogenic flask - a cold boot attack. He isn't selling RAM, he's selling data.
That is a pretty nitpicking reason to say it has not aged well. Hamlet doesn’t have cell phones yet I think it’s an excellent play. Even though a quick FaceTime would’ve averted a tragedy.
Here’s another fun nitpick from Neuromancer. The opening line says the sky was the color of a TV tuned to a dead channel, which in Gibson’s day would have been analog static that looks like a messy grey from across the room.
But later, with computerized channel tuning, a dead channel was shown as a screen of solid bright blue, and even later, solid black. So different generations of kids have grown up with very different mental images of the background lighting for the opening of Neuromancer.
1 reply →
Hamlet is set in (what is currently) the past. It is self consistent. Neuromancer is set partially in the future, and partially what is our past. The inconsistency is what throws people. It can be a good book, but people might still find those elements jarring.
So many plots in Seinfeld would have been trivially solved with cellphones. Get separated in the parking garage? Call each other. Need help carrying an armoire? Call each other. Trying to meet up at the movie theater? Call each other.
2 replies →
[flagged]
8 replies →
Side note. It is sort of the unwritten rule in TV and Film. Don't highlight technology unless absolutely necessary. it can age it considerably.
A few weeks back I had '30 Rock' on, it was funny seeing everyone with flip phones, it wasn't highlighted but it just makes you realise the age of it all.
I always just find stuff like that charming. I love seeing relics of the time in which something was made. I love seeing dated pop culture references, it's one of my favorite things about engaging with old media.
I don't keep anything on my phone anymore. I use gadzorp.
Same with monetary amounts.
Movies in 1950s: "For this heist you will receive a fantastical sum of $100,000."
Nowadays: "$100,000 is a mediocre downpayment in the suburbs."
It's funny that some of the more hand-wavy books are closer to what is happening.
If I recall correctly, Gibson had never even used a computer at the time of writing Neuromancer, so that's perhaps not shocking.
this is my issue with cyberpunk “literature”
it’s a genre written by people who barely understand technology and consumed by even more luddite types.
it’s all uninformed fear mongering
7 replies →
Can't you just read "3 MB of RAM" as a large amount of some scarce tech resource and move on?
What if you got a on-chip compression algorithm so advanced that you can fit a world in a few MB and now with corporations controlling memory distribution, 3MB of high compression memory is highly valuable in the black market.
"High compression memory" is immersion-breaking magic. There are hard mathematical limits to how much you can compress things (Kolmogorov complexity), and every lossless compression algorithm necessarily increases the size of some inputs (pigeonhole principle). It's much simpler to assume that "M" is slang for some bigger unit.
We already have the real life example of people using "mega" (10^6) as slang for "mebi" (2^20).
1 reply →
It's funny... I enjoyed Neuromancer, although I didn't read it until about 15 years ago.
And, yeah, the memory thing hasn't aged well. Thing is, 1984 was a funny time in computing, particularly when you consider the kind of computers normal people had access to.
At that point even things like PCs and the new Mac had 128 or 256K of RAM[0], so I get that 3MB must have seemed like an ocean of memory at the time. And, realistically, more than 1MB of RAM in machines you'd typically see sat at home or on a desktop was uncommon until the beginning of the 1990s.
And, although Moore's law had been around since 1965 it's hard to know how aware people outside of specialist circles would have been of it in 1984.
I suppose Gibson must have done some pretty in depth research for Neuromancer, right? But the memory thing is sort of ancillary to the story, so how much would he really have focussed on that? Probably not much.
And then do you really want to harshly judge the book on that one slightly laughable thing, in other ways, it was incredibly forward looking and almost prophetic? Doesn't seem right.
I think the sensible position is you have to let it slide and see it as a possible alternative future that never quite came to pass in that way but that which we can see strong echoes and foreshadowings of even still.
[0] In 1984 microcomputers, as opposed to, cough, "serious" computers like the PC and Mac, with 128K of RAM were still very new, with 32 - 64K being the entry level, and if you had one with 128K you were king of the hill. 128K in 1984 seemed like a ton of memory to most of us, but it's worth bearing in mind that only a handful of years before computers like the ZX81, which had only 1K of RAM, were the common entry level, so the progression was already clear if you looked at the situation in the right way, but you had to have been paying attention for a while to have noticed. I remember the first time I used a machine with 4MB of RAM in, maybe, 1990 - an Archimedes at school - and feeling like it was just this absolutely inexhaustible ocean of memory. In 1984 3MB would have felt almost inconceivably huge unless you were in the high performance computing, or maybe the mainframe, worlds.
> But then Gibson must have done some pretty in depth research for Neuromancer, right?
Isn't he on record that his documentation was listening to techies talking shop in bars?
> And then do you really want to harshly judge the book on that one slightly laughable thing about memory when, in other ways, it was incredibly forward looking and almost prophetic.
He seems to understand humans. Gibson's world and Brunner's Stand on Zanzibar are imo the most "prophetic" sf books written so far.
2 replies →
It's sort of a cool idea. "Pre-RAM" without the tracking/AI integration so it can be used for clandestine activities in a dystopian future.
It's 3MB of "hot" RAM, IIRC. Makes sense.
I've always wanted to set up automation that updates the text for the book every so often to preserve original intent, by changing the language around quantities to <whatever would make a contemporary reader be impressed>
Replace "megabyte" with "exabyte" or whatever.
As an aside, there's a great essay I think in Metamagical Themas about the fuzzy task of translating literary works, which takes up the question of whether it would be valid and how to translate say Dickens into French, by relocating from London to Paris.
Part of the premise is that this is an impossible task because the referential systems are not truly analogous; the cultures aren't identical so some concepts literally have no direct translation...
...in Neuromancer the mere existence of a bank of pay phones has stood out as one of those things which even my little updater would break hard.
Also related: I've seen a slowly increasing number of complaints in book forums about text from relatively recent novels being silently updated, to change references from e.g. Myspace to Tik Tok or whatever, to try to keep the text feeling current... a perilous slope.
And the bank of pay phones is one of the coolest little scenes in the book.
Put me on team “don’t update the text.”
Books are written in their contemporary cultural contexts. LeGuin said speculative fiction is about today even if the story is set in the future, and I agree.
So when you read Neuromancer, in some ways you’re reading a book that is about the 1980s in the U.S. So there are more fundamental anachronisms than just RAM and payphones, like a belief that east Asia had a lock on the future of advanced technology. Or that punk culture was cool and edgy and would endure.
You'd be stuck on how to update "the color of a TV tuned to a dead channel". Was never sure if that meant a starry sky or a bright blue one.
1 reply →
Why not go directly to the source article that has a lot more details?
https://www.theregister.com/systems/2026/06/29/zuck-saves-me...
Source paper linked is https://aisystemcodesign.github.io/papers/isca26/vistara_cam...
From a quick skim, you could think of this as roughly equivalent to shoving a large amount of DDR4 on a PCIe card and using it as a swap space. It's more sophisticated (see CXL protocol), but that gives you an idea of the tradeoffs. It seems there is some OS-level support for moving hot/cold pages between the main fast DRAM and the expansion higher latency DRAM.
It's a very valid point that DRAM has a fairly long lifetime and contains significant embedded carbon emissions, as well as the current availability crisis of new DRAM.
> and contains significant embedded carbon emissions
Hi - thanks for the insightful comment - could you please expand on the above?
Genuinely curious :)
5 replies →
I’ve wanted this for a long time and it seems to reemerge during RAM boom cycles and then disappear during busts.
I have 32GB of DDR3 that would be great for scratch space or cache of i could throw it on a card.
It is amazing how this is attributed to "zuck", like he actually knows these things.
That's just El Reg being El Reg (the Zuck-buck rhyme was apparently too good to pass up). But it's a far cry from their glory days, when they coined nicknames like "The Beast of Redmond" for Microsoft or "Chipzilla" for Intel...
> Why not go directly to the source article
Which seems to be the sister site of Register; https://www.blocksandfiles.com/architecture/2026/06/26/panmn...
The writers name is Maxwell Cooter, which made me giggle.
I have always wondered why there was never a big market[1] for "cheap PCI/PCI-X/PCI-e card you can stick a boatload of your old/surplus/n-generation old simms/dimms on and use as swap/slow memory/ram disk/etc". It's rare you can populate a motherboard with a full address space full of 'new' memory, and you can teach kernels to prefer some memory to others because of speed[2], so it seems like a no-brainer.
I seem to remember the market for doing similar with flash got neutered over patent issues, but I can't recall the details. And flash cache did end up being a market, at least for bigger players. Maybe something similar happened here, or maybe it just hit a niche I cared about at the time?
[1] I know there were a handful of products in this space, but my impression is they never really took off. I could be wrong. [2] Definitely can in NetBSD; I've done it for archs like VMEbus where it's common to have a small, fast on board memory and much slower, often larger memory out on the bus. I assume this sort of thing is enabled in Linux by the work to support NUMA, but I've never looked into it.
There was never a big market for it because new memory was not prohibitively expensive in comparison to the cost, risk, and limitations of using old memory in a new server. That is not the case now, so people are looking at the idea again.
That's a fair take and likely the answer.
I would counter tho that 1) this isn't the first time there's been a memory price/supply crunch, and "I've got a drawer full of last gen memory I can't use" is kinduva IT cliche, and 2) 'more memory' has always been a pain point, especially with industry practices like chipsets only supporting relatively small physical memory relative to address space (e.g. all those Intel LGA775 chipsets that capped at 4 or 8GB). Oh, and 2a) 'faster disk' has always been a pain point...
But, yeah...obviously my impression of things doesn't match market reality.
1 reply →
Not exactly the same but I'm building a NAS box with old parts. Most of my spare RAM sticks are laptop DDR4 SODIMMs. There are SODIMM -> desktop DIMM adapters... it did not go well. The system would boot 1 out of 5 times. No adjustment of memory speed settings would make the system stable.
That's not surprising. At the speed DDR4 runs, PCB design has to account for all sorts of weird effects. Adding the additional traces on the adapter probably pushes all sorts of timing over acceptable thresholds. There's a reason one of the big textbooks in the area is subtitled "A Handbook of Black Magic". [1]
[1] https://www.amazon.com/High-Speed-Digital-Design-Handbook/dp...
What board are you using that supports SODIMM? I’ve got 64GB I want to put in a workstation but I’m not finding the right solution.
30 years ago I remember cards like this to convert 4 30-pin SIMM modules to fit in a 72-pin SIMM slot.
https://www.ebay.com/itm/383521792853
Oh, yeah...I remember those. I prolly have one in a box someplace. They were pretty terrible. :-)
Old generation DIMMs are not that much cheaper and supply is limited as old generation are taken out of production to free capacity for newer chips. DDR2 is already more expensive than DDR3, likely because it is no longer produced but there is still demand (to replace/upgrade memory in older hardware).
My understanding (which could easily be wrong), is the big difference today is CXL which adds cache coherency on top of pci-e.
Without cache coherency, you have to be more careful about how you use the memory and the performance story is complex. Ram over CXL is going to have worse perf than ram on the cpu memory controller, but there shouldn't be any big gotchas.
You could give it a similar structure to CPU cache or a giant swap file. Have a giant blob of memory on the PCI RAM, that feeds the on board smaller motherboard RAM as needed. It might just work.
Nvme drives already max out the 4x pcie lanes they get. You'd basically need to use the GPU slot to do better and even then you could do it with SSDs. M.2 break out cards are pretty common.
However unless we're taking 3DXpoint which was discontinued flash wears out. But yeah sticking an old optane nvme on a breakout card accomplishes much the same thing.
Cost for me. I wanted a Gigabyte I-ram but it was too expensive when I only had one 512MB DDR stick after upgrading to 4GB of DDR2. I bought a 60GB Sandforce SSD that fulfilled that speed gap.
Funny how the idea only seems to become practical once you have a literal warehouse problem worth of old RAM
Honestly DIMMs don't really fit on PCI cards and a boatload definitely doesn't fit.
I have always wondered why there was never a big market[1] for "cheap PCI/PCI-X/PCI-e card you can stick a boatload of your old/surplus/n-generation old simms/dimms on and use as swap/slow memory/ram disk/etc"
Reminds me of the days of JBOD arrays. Mac OS X had built-in support for it.
JBOR?
JBOD arrays are still a thing. They've even evolved a bit (see: UnRaid).
They used to exist
Go back and read beyond the first sentence; you'll see I said exactly that.
2 replies →
There are standard product CXL memory expander chips if you don't want to design a custom chip.
https://www.marvell.com/products/cxl.html
there's multiple vendors even https://www.asteralabs.com/products/leo-cxl-smart-memory-con...
The interesting part of this "RAM crisis" is similar to other fields where a problem results multiple parties looking for alternative solutions.
This yields for exciting ideas or workarounds that might result a post-crisis memory boom (hopefully) also for local machines.
1. Lowest, Apple is evaluating new Chinese manufacturer which means change of supply demand if indeed it has reasonable QA. (https://www.ft.com/content/f4ac5c92-03be-4499-b16a-017a7e9ee...)
2. Companies tries to workaround performance - suddenly single channel is 'ok' ? :) (https://www.gigabyte.com/press/news/2403)
> suddenly single channel is 'ok'
Single channel RAM surely beats any disk-based swap.
of course, but I was under the impression the real shortage is RAM mostly.
1 reply →
I would love it if we started designing software with hardware constraints in mind again.
Consumer hardware costs are external. We are in the era of ignoring externalized costs, for example, Windows 11 hardware requirements, social media harm, and mainstreaming of gambling. Maybe the pendulum will swing back in the future, but today, line-going-up is the prime directive.
We do already, if it ooms at 32g I have to prompt again /s
Necessity is the mother of invention, after all. (One of the oldest abstract concepts in intellectual thought, I suspect.)
There is a tight resource starvation/motivation loop — the demand put on RAM and SSD and GPUs by the largest frontier models is a direct motivation to make smaller LLMs. Like an evolutionary pressure making animals smaller and more food-efficient.
These smaller models, once successful, are still likely to consume more RAM and SSD and GPUs than any other application short of high quality video processing itself (the smaller LLMs and higher end video processing seem to have about the same needs). But the resources would distribute through the market more traditionally, leading to less insane cycles.
So it seems to me that the way out of the RAM/SSD price cycle crisis that manufacturers are in — where the price fluctuates between high and low due to supply constraints and then oversupply from new production capacity - is for them to fund research into smaller LLMs. They'll still sell essentially the same amount of product. Maybe more.
Necessity is the mother of invention.
It’d be nice if there were a consumer version of this. I have plenty of old RAM.
Gigabyte had a ram disk addin card years ago. Not exactly the same but since it's presented as a storage device you could use it as OS swap space.
https://en.wikipedia.org/wiki/I-RAM
CXL Vistara reminds me a bit of the AST Rampage 286 memory expansion ISA card I had in my 286 back in the day, as a kid. Things go in circles, I guess.
observation that I've noticed recently: what's with wikipedia downsizing the hell out of images site wide? Every image I look at is garbage and I have to dig through multiple links to find the original.
2 replies →
Adding the RAM to the system this way isn’t exactly like expanding the main system RAM. The RAM is connected over a PCIe type link so it doesn’t behave like the primary RAM.
It’s better for server farms where engineers can customize and tune for an architecture like this.
There have been some cards that use RAM as a storage device. They were never popular because having to set it up as a disk had very limited use cases.
To someone not familiar with CXL that still gives the wrong impression. As far as I have seen, CXL is supposed to be cache coherent, and should require less invasive rework (if any at all) of applications to take advantage of it; that's part of the enablement of memory disaggregation that CXL is pushing towards (similar to the storage disaggregation push a decade or so ago).
1 reply →
The sensibility is a bit different since consumer systems don't really have much bandwidth back to the CPU. Given the current resale prices of DDR4, might as well just sell it and get some NVMe drives.
Oh good, so now the prices of used RAM can go through the roof, too.
Man, I hope the price of a PC goes back to being equivalent to a car. So many monetization opportunities there.
It already is. Used 5yo DDR4 is currently priced more than it did new 5yrs ago.
ServeTheHome already reported on CLX memory expansion controllers back in December: https://www.servethehome.com/hyper-scalers-are-using-cxl-to-...
It will be interesting to see what happens to the consumer electronics market the next few years. Companies are right now gambling that consumers will pay extra because of RAM shortages. I suspect with the cost of everything else rising as well, a large portion of consumers (remember, HN, not everyone makes tech money) will just not be buying new devices for a bit.
I think I'm in this boat. I'm just choosing to interact with technology less. Everything has just gotten more hostile that it has reached a tipping point.
I just upgraded from a Lenovo T400 to a Carbon X1 4th gen for $100. I can ride out this storm longer than the storm can continue.
My needs are much lower than what the market demands.
I'm with you on this one. I'll be using my 11th gen i5 and GTX3080 for the foreseeable future. The 'hostility' you mention puts me off doing so more than price of an upgrade.
From the paper:
"Our CXL solution achieves substantial gains for diverse workloads, including up to a 25% reduction in server count for disaggregated ML inference"
How does using worse RAM result in 25% reduction of server count for given workloads?
CXL is adding "slow" RAM over PCIe, basically. Not replacing.
Because it's used in addition to, not in place of, the better RAM.
Thank you, LLM / AI dorks! I cannot even sensibly finance a proper homelab firewall / router these days, thanks to you and your agentic shit.
Yeah seriously. Thanks for ruining everything I love.
ScholarlyArticle: "Vistara: Making CXL Real—Full Path from ASIC Design and OS Support to Hyperscale Deployment" (2026) https://aisystemcodesign.github.io/papers/isca26/vistara_cam...
TIL there are 2x 2.5GbE PCI-E HAT adapters for Pi 5.
How to attach RAM to the new NVLink/UALink fiber buses?
Literally the "new old thing":
https://www.andysarcade.net/store2/all-other-stuff/vintage-c...
It reminded me more about these kind of things actually: https://www.ebay.com/itm/277636244509 (ISA RAM Expansion Boards from the PC/XT era)
I wonder if it would ever start to make sense to burn an AI model into ROM, replacing a large portion of an inference machine's RAM with ROM. (Probably not, since I'm sure those machines do dual-duty and run training when the inference workload slows down)
That's the idea behind Taalas (https://taalas.com), except as silicon rather than ROM. They run a demo at https://chatjimmy.ai/ which serves an old open weights model (Llama 3.1 8B) at something like 15,000 tokens per second.
Companies are building chips specialized for inference, so dual use for training isn't necessarily a consideration, but there are other considerations such as:
Weights need to be loaded into the accelerator's processor fast, which means they need to be physically adjacent to it, but there is limited physical space for that - not enough to fit the all the weights of a 1T+ param model, so weights get loaded into VRAM dynamically according to what part of the model is being run.
ROM (I guess we're talking Flash memory) can be dense, since it is built vertically - many hundreds of layers, but this comes at the cost of poor performance, so even if you could fit enough ROM next to the processor it would not be fast enough.
The impressive part is not really reusing old RAM, its making the economics work despite the extra chip, software support and operational complexity
In the future, hardware is only for big companies to own. At least it seems we're heading that way.
> In the future, hardware is only for big companies to own. At least it seems we're heading that way.
China is desperate to sell anything to... everyone. If there's a market, they'll eventually be there to fill it.
It took them decades for cars, but now they did it.
For RAM, CXMT went from 20 000 wafers per month to... 240 000 wafers per month in something like two years. And they're extending capacity massively now. It's a company only 10 years old.
The market is there and China shall flood it: that's how they operate with everything.
At some point they'll probably even come with GPUs that shall do 80% of the job for 20% of the price.
Just like you can buy chinese server motherboards at 1/5th the price of a SuperMicro one today.
So I'm not sure hardware is going to be only for big companies: China is going to put pressure on the OpenAI and Anthropic of this world locking all the RAM / SSDs / chips of this world.
> At some point they'll probably even come with GPUs that ...
I'd already like to play with the huawei card but everything is so overpriced at present and I don't think they sell those on the consumer market to begin with.
I think there's a great deal of underestimation of China's manufacturing. Granted, wafers are a totally different thing than any of the other industries they've dominated for cheap, but I certainly wouldn't count them out.
I've recently gotten into fountain pens. Sure, a $7 Jinhao or $15 Hongdian pen isn't going to write quite as nice as a $200+ pen, but they're about 80% of the way there, and you can buy tons of them for the cost of a single more expensive pen. Plus, some models will accept Western nibs just fine which means you're buying a cheap barrel and assembling a much higher quality product for almost pennies on the dollar.
One would do well not to underestimate their ability to fill markets. It may take years, but it will happen.
Can you buy Chinese cars without the Multimedia/GPS computer and "phone home" system?
3 replies →
Have you seen the talk: The Coming War on General Computation?
No, but I'm sure I'll agree with a lot of things in it.
With regards to RAM price I never understood the following: A 16GB RAM stick has 16*8=128 billion bits, with 1 transistor per bit, thats still 128B, yet its supposed to cost like $60 before the price hikes? In contrast, a 5090 GPU was $2000 (true it has RAM, but you're paying for the GPU ASIC really, I guess the rest of the GPU was less than $500), it had 93B transistors.
GPU transistors are smaller due to the more advanced process node (cost per transistor metrics aren't really clear, if they improve on advanced node or not, but I'd say they get cheaper as they get smaller, as technology costs are amortized).
I'm sure both RAM and logic use a process that is quite similar in both inputs and manufacturing steps. So while RAM is a commodity product, this insane price difference didn't make any sense.
So I guess when those fundamental inputs become a constraint, it would make sense for $/transistor move closer for both, which is a massive hike for RAM.
Chip fabrication processes are not fungible: GPUs and CPUs might be made on roughly the same process, but DRAM is not (flash is a different process again, as is power electronics, analog electronics, MEMS, etc. And even within those broader categories there are different variations). While there are some overlaps in machines and techniques, a fab set up for one cannot generally switch to the other, and the economics of each process can also be drastically different.
You're not the first person to say so (and I don't mean to dispute it), but I have never been able to find a clear answer for /why/ those processes are incompatible.
Is it built in different silicon, is it physical steps that's incompatible (ie its actually incompatible), is it different physical preparations that needs to be made (making it economically infeasible to combine)
I cannot help but wonder, even if the answer doesn't change anything in my life.
9 replies →
Consider that the R&D cost of the GPU (>>> RAM) must be amortized across total units sold (<<< RAM).
RAM is literally copy-paste of the same circuit over and over, you're trying to compare cost to produce million AK-47 with 1 carrier
and cost per transistor stopped decreasing at ~20-30nm, now small nodes are targetting energy efficiency (and thus performance, since heat is the main limiter)
On top of everything said, 5090 die size is 10x than typical DDR5 die size. One RAM module is 8-16 dies, so you do get more silicon in the end, but larger dies are extremely expensive to produce due to sharply decreasing yields.
The thing that defines performance of DRAM is AFAIK the capacitor of the bit cells and not the transistor driving it. And also AFAIK the process to create those capacitors is quite unique to DRAM, so you can't just go and use a "logic" process unchanged and produce DRAMs.
newer process node are smaller but very expensive compared to mature ones, each wafer from TSMC latest process is costly and with lower yield due to GPU large die size (+700mm2 compared to around 60mm2 per DRAM die)
RAM is a commodity. It has much less moat to prevent competitions. When the rams flood the market that's when the bubble ends, until the next cycle arrives. Processors are much harder to design and commoditize.
Why would you expect smaller transistors to be cheaper?
Because you are paying for silicon, and processes, not transistors. Wafers have a certain cost, and litographic processes can illuminate a certain X mm2 of dies in an hour. If a transistor gets smaller, more of them fit in a certain area.
Granted the machines that make them become more expensive, but that's capital expenditure, which gets amortized as time goes on.
So there are two forces here working against each other.
1 reply →
That is what Moore's Law said.
4 replies →
> So while RAM is a commodity product, this insane price difference didn't make any sense.
Supply and demand coupled with the fact that a RAM fab can't (trivially) output compute chips, and vice versa, a compute fab can't output RAM. It's two completely different supply chains.
A GPU Transistor is a lot more complicated than a RAM transistor and the size of these are quite different too. Bleeding edge vs. a known process with know machines and written off machines.
Also you calculate in the machine cost and R&D.
RAM hiked because the demand spiked and these companies are now in power. Before apple and other companies told them the prices and had hardly any money for investment.
I am have an old Pentium 4 with RDRAM, think I could sell it to them? I think it has like 256MB. Haven't turned it on in awhile. Hope the first 640KB still work.
This sounds somewhat similar to IBM's Centaur used on the POWER9/10/11. Some of the POWER9 hardware people ended up at Meta so might be related.
It'd be interested to see how one could leverage all the DDR3 ECC that they may have laying about; maybe an overseas shopping site has these boards available? Would DDR3 be as fast or faster than an SSD?
top of the line SSDs now eclipse DDR3 throughput, but DDR3 should retain a large edge in latency of orders of magnitude
absolutely no idea how useful any of that would be and what kind of latency degradation going through whatever adapter would cause
WE need to learn to use computing resources more efficently. Use RAM more efficently.Todays software just squanders computing resources.- like RAM
Not terribly exciting at 1/10th the bandwidth and double the latency. That's a heavy price to pay to use old DDR4 memory.
Probably good enough for memcached type applications
Wonder if Intel optain will would have made a huge comeback.
Ram drives are making a comeback :)
Time to dust off my DDRDrive
If they grow desperate I have GBs of DDR2 AND DDR3 in a drawer.
I've got some RDRAM-- still waiting for the RAMBUS revolution, any day now
Make sure to charge them double!
You've got a friend request form Zuck
There is already a data center oversupply. xAI rents out colossus and Meta also rents out capacity.
if not the prices, no one would have implemented this in large scale solution..
It makes me happy hearing that Meta has been relegated to sloppy seconds.
how the mighty have fallen! Can't wait to see
Who would have thought that the hardware I own didn't go down in value for the first time in my life but almost doubled in value.
Supply-demand economics really went awry in the age of chasing agi
Now to rip RAM off PCs being sent to the landfills.
I fail to see why you couldn't also literally just stack RAM chips themselves on top of each other. If it works for stupid consoles (see NES/Dreamcast/PS1/PS2 hacking of recent), PCs should be no different.
I am waiting for folks to confirm that pulling them off machines designated for the landfills was a thing.
Url changed from https://www.networkworld.com/article/4192827/meta-reuses-old..., which points to this. (I've linked to the paper in the toptext as well.)
Submitters: "Please submit the original source. If a post reports on something found on another site, submit the latter." - https://news.ycombinator.com/newsguidelines.html