Metraipolis › archive › dc_cb8bb99268f5
MetaDB: Putting File-System Structure Back into Distributed Databases — arXiv:2610.32247
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.32247
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-26T19:54:42.232Z
- SHA-256 of the archived text
ec944e238e85510d0cc3572a3562f8ebb4866ca00431df1a2cad8f30f017fa28- Stable URL of this page
https://metraipolis.com/documento/dc_cb8bb99268f5
How to cite
MetaDB: Putting File-System Structure Back into Distributed Databases — arXiv:2610.32247 — arXiv (en), https://arxiv.org/abs/2610.32247 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cb8bb99268f5 (Metraipolis archive dc_cb8bb99268f5, sha256 ec944e238e85510d0cc3572a3562f8ebb4866ca00431df1a2cad8f30f017fa28)
BibTeX
@misc{dccb8bb99268f5,
title = {MetaDB: Putting File-System Structure Back into Distributed Databases — arXiv:2610.32247},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.32247},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cb8bb99268f5; sha256 ec944e238e85510d0cc3572a3562f8ebb4866ca00431df1a2cad8f30f017fa28},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.32247. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Storage systems in production increasingly rely on distributed databases for metadata management, yet our experience at Baidu shows that treating a general-purpose database as a black box leaves substantial performance and efficiency untapped. File-system metadata exhibits directory-tree locality, differentiated consistency requirements, and clustered deletions that general-purpose databases largely ignore. We thus propose MetaDB, a metadata-aware distributed transactional database that serves as the shared metadata layer for Baidu's large-scale storage systems. MetaDB addresses three key mismatches between general-purpose databases and metadata workloads: Data Model Mismatch, Transaction Model Mismatch, and Storage Implementation Mismatch. To bridge these gaps, MetaDB combines (1) tree-topology-induced sharding and execution, (2) semantic-aware off-critical-path processing, and (3) a lifecycle-aware storage engine with in-memory MVCC and locality-aware compaction. MetaDB achieves 2.2x-4.1x the throughput of our previous general-purpose database and halves the number of metadata machines. Deployed in production at Baidu for over five years, MetaDB unifies metadata management across the company's storage fleet and scales to over 1,000 metadata nodes, tens of trillions of metadata records, and exabyte-scale storage in a single cluster. Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno). Categorie arXiv: Databases (cs.DB) ; Distributed, Parallel, and Cluster Computing (cs.DC). Pagina del lavoro: https://arxiv.org/abs/2610.32247 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-26T19:46:03.304Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).