Comment by tempay
6 days ago
No they're not. At CERN physics data is on:
* CTA for tape storage: https://cta.docs.cern.ch/v5/
* EOS for disk storage: https://eos-web.web.cern.ch/eos-web/
There is a large CEPH cluster as well but that isn't really used for physics data.
More specifically, from the EOS page:
> Data is stored natively in XFS filesystems on hard disks or SSDs or on virtualised back-end storage (e.g. RADOS block devices) or distributed filesystems like Lustre or CephFS.
Interesting, they used to be the largest ZFS user.
Hard to Google for it without getting AI slop on it, but apparently they built their own stack in 2019.
Not sure I like their solution, "Meta-data is persisted in RocksDB databases using a proprietary KV store called QuarkDB." Unless QuarkDB has magically removed RocksDB's amazing ability to corrupt and lose data frequently, this whole thing sounds like a bad idea.
Also, their data is not stored on any one system (local XFS, Ceph, Lustre), a recipe for disaster.
There's at least 3 copies of all the data and possibly more spread around the world at different Tier1 and Tier2 centers. The T1 and T2 centers are very likely using different filesystems and everything is checksummed pretty much at every major step (data generation, before and after any network transfers, before analysis, etc).
The problem with RocksDB is Facebook uses it, and has a similar system that replicates data across the world... and still faces unrecoverable data loss because of RocksDB.