Comment by arjie
5 days ago
This can’t be right. 2 PB of flash is like $200k. It’s within reach of many individuals. Then again I guess you don’t need that much storage so maybe it is.
5 days ago
This can’t be right. 2 PB of flash is like $200k. It’s within reach of many individuals. Then again I guess you don’t need that much storage so maybe it is.
Also my first thought: "Is that... a lot?"
You can put 6PB (244TB * 24) into a single box these days.
More like $1M at current prices at this scale / level of performance.
If you go with HDD arrays probably $50k
Boy pricing is pretty nuts these days. I have half a petabyte in Seagate enterprise drives myself and I didn’t pay anything close to that to acquire it. Such a pity about the flash storage. 2 years ago we built 200 TiB or something of flash using Samsung PM1633 or something and it was a fraction of the cost per gigabyte that $1m would imply.
We're in the boom phase of the cycle. The bust on these chips always comes.
Your numbers are a little off but the point remains- 2PB is nothing, not newsworthy imo. What’s special about this?
What's special about it is not the flash but training an LLM based on the content, much of which is still in copyright and which the library has restrictions on how they are allowed to use (irrespective of the legal position of training on it) and which required an agreement with the copyright holders.