Why That 36 TB LTO-9 Cartridge with 63 Million Files Might Be Your Next “Great Disaster Recovery Story” When the Versity audit output landed on my desktop last quarter I nearly spilled my coffee. Here was an LTO-9 tape with roughly 63 million files and about 36 terabytes of data (assuming a 2:1 compression ratio), representing nearly 15 percent of the curatorial collection for that groups archival space. A second tape clocked in at over 40 million files, and the rest of the collection sprawled across another 20+ cartridges summing to roughly 150 terabytes and over 415 million individual objects. The good news? The tapes are readable. The bad news? At the current throughput and file size distribution, recalling all the data would take months — over half a year — due to files being small in size and sequential media physics eating throughput for lunch. When the Versity audit output landed on my desktop last quarter I nearly spilled my coffee. Here was an LTO-9 tape with roughly 63 million files and about 36 terabytes of data (assuming a 2:1 compression ratio), representing nearly 15 percent of the curatorial collection for that groups archival space. A second tape clocked in at over 40 million files, and the rest of the collection sprawled across another 20+ cartridges summing to roughly 150 terabytes and over 415 million individual objects. The good news? The tapes are readable. The bad news? At the current throughput and file size distribution, recalling all the data would take months — over half a year — due to files being small in size and sequential media physics eating throughput for lunch. I’ve lived through tape generations from 4 mm DAT to LTO-9 and beyond. I’ve architected library farms, shepherded migrations, wrangled object counts most people never want to see. And I’m on record saying this clearly: a single, monolithic preservation object — especially tape — can become a liability unless we rethink how we store, protect, and manage massive volumes of small objects on sequential media. never a single, monolithic preservation object — especially tape — can become a liability can Let’s unpack why, and chart a course forward. Why This Matters Why This Matters Tape still plays in the big leagues. Magnetic tape technologies like Linear Tape-Open (LTO) are fundamental to archival, backup, and long-term retention because they deliver cost-effective, high-capacity storage with extremely low energy use and high media life. Tape’s bit error rate and predicted longevity remain competitive compared with disk, giving organizations a viable path for “store it forever” data sets. [Ultrium LTO] Ultrium LTO But those strengths obscure a weakness: tape is sequential and singular in structure. Unlike clustered disk/object stores where metadata engines and distributed erasure coding are table stakes, tape is often treated — by software and ops alike — as a black box: carve the content in, forget it, hope for the best. That model breaks down at scale. tape is sequential and singular in structure Here’s the core tension: Growing data volumes with high object counts magnify recall and integrity complexity.
Traditional tape protection strategies were never designed for objects at the scale of hundreds of millions on a single medium.
Modern expectations for availability, recoverability, and automation are not aligned with tape’s sequential, linear nature. Growing data volumes with high object counts magnify recall and integrity complexity. Growing data volumes Traditional tape protection strategies were never designed for objects at the scale of hundreds of millions on a single medium. Traditional tape protection strategies Modern expectations for availability, recoverability, and automation are not aligned with tape’s sequential, linear nature. Modern expectations for availability, recoverability, and automation This is not a hypothetical edge case anymore. You are running into this in production. What People Usually Miss What People Usually Miss Let’s dismantle the common misconceptions that get people blindsided: A. “Tape capacity growth means tape can store anything.” A. “Tape capacity growth means tape can store anything.” Yes, an LTO-10 tape can hold multi-terabytes (specced higher than LTO-9). But raw capacity does not mitigate recoverability constraints. Tape throughput and capacity metrics gloss over seek delays, threading latencies, and multiple file overhead — which dominate recall time when billions of small objects are involved. can hold not seek delays, threading latencies, and multiple file overhead The fact that tape can physically record 100 TB doesn’t address how long, expensive, and fragile it is to read back those 100 TB if you didn’t design for scale at the file/object level. B. “Sequential media doesn’t need modern protection.” B. “Sequential media doesn’t need modern protection.” Almost every data protection playbook for tape boils down to: Copy the tape (create duplicates)
Store them separately
Hope neither goes bad Copy the tape (create duplicates) Copy the tape Store them separately Store them separately Hope neither goes bad Hope neither goes bad And you pray — no checksums? no cross-verification? no continuous protection? This is the mid-1990s mindset resurfacing in 2025. Meanwhile, in distributed object storage on disk, erasure coding and versioning are standard. Tape has barely scratched the surface of applying those concepts at scale. barely scratched the surface scale C. “If the tape reads today, it’ll read tomorrow.” C. “If the tape reads today, it’ll read tomorrow.” Tape media fidelity is high, but it is not infallible. LTO and other magnetic tapes have excellent empirical reliability numbers versus disk — but that’s at the media level, not at the object composition level. When you pack tens of millions of objects with tiny average sizes onto a single medium, you are aggregating risk. A localized media defect or servo issue can threaten a huge fraction of a dataset. infallible object composition D. “Backups are backups.” D. “Backups are backups.” Traditional backup vs. preservation cloud mythology blinds teams. Tape preservation data is not interchangeable with backup data in requirements. The expectation for recall time and access patterns is fundamentally different. Tape preservation jobs are read-dominant, and the failure modes cannot be treated like disk arrays or cloud object stores. is not interchangeable with backup data read-dominant The Core Risk Vector: Single Medium, Exponential Objects The Core Risk Vector: Single Medium, Exponential Objects Let’s quantify why a 100 TB archive with 400 million objects becomes a liability: Sequential access dominates performance. Average seek times multiply with small objects — the throughput often collapses from tens of MB/s to single digits or worse as you skip around tiny files.
Metadata overhead explodes. Libraries and management systems struggle to keep indexes for tens/hundreds of millions of entries.
Recall latency becomes unbounded. Weeks or months to restore results in stale, unusable data — especially in compliance or legal holds.
Repair cycles become impractical. If the tape goes bad, you are looking at expensive forensic restores or reconstructs from secondary copies. Sequential access dominates performance. Average seek times multiply with small objects — the throughput often collapses from tens of MB/s to single digits or worse as you skip around tiny files. Sequential access dominates performance. single digits Metadata overhead explodes. Libraries and management systems struggle to keep indexes for tens/hundreds of millions of entries. Metadata overhead explodes. Recall latency becomes unbounded. Weeks or months to restore results in stale, unusable data — especially in compliance or legal holds. Recall latency becomes unbounded. Repair cycles become impractical. If the tape goes bad, you are looking at expensive forensic restores or reconstructs from secondary copies. Repair cycles become impractical. This is where the previously inviolable “tape integrity” assumption fails: at scale, risk is amplified by object density. risk is amplified by object density How Do We Build a Resilient Tape Preservation Environment? The Pro Position How Do We Build a Resilient Tape Preservation Environment? The Pro Position I am unequivocally in the “tape preservation can be safe *if re-engineered” camp. The traditional model of tape + copies is short sighted for high-density object collections. Here’s what I advocate: A. On-Tape Erasure Coding A. On-Tape Erasure Coding Apply erasure coding directly within the cartridge’s data organization. Instead of recording objects linearly with simple error correction, embed Reed–Solomon or similar erasure codes across data stripes within the cartridge. erasure coding directly within the cartridge’s data organization This is not theoretical — patents exist for erasure coding on magnetic tapes using dual Reed–Solomon codes to reduce effective bit error rates and enable recovery from localized loss. [Patents] [Patents That changes the model from “read the whole tape to find errors” to “reconstruct lost stripes from intact parity blocks.” B. Cross-Tape Redundant Arrays of Independent Tapes (RAIT) B. Cross-Tape Redundant Arrays of Independent Tapes (RAIT) The concept of redundant arrays across tapes — sometimes referenced as RAIL/RAIT (Redundant Array of Independent Libraries/Tapes) — extends erasure coding across multiple cartridges. Instead of duplication, use parity across tapes to allow recovery of data if one tape fails entirely.ThinkMind across multiple cartridges one tape fails entirely ThinkMind Implementing RAIT means: Distributed parity across tapes
Intelligent catalog that knows which tape holds which stripes
Reconstruction algorithms that can rebuild without loading every tape Distributed parity across tapes Intelligent catalog that knows which tape holds which stripes Reconstruction algorithms that can rebuild without loading every tape This is modern datacenter thinking applied to tape. C. Holistic Tape Media Protection C. Holistic Tape Media Protection A single tape cartridge acts as a signal device with finite reliability. Embedded servo tracks, magnetization drift, and media wear are real. We need: signal device with finite reliability On-tape checksums per object (not just block)
Continuous health tracking from library metadata
Proactive tape replacement based on usage and health On-tape checksums per object (not just block) Continuous health tracking from library metadata Proactive tape replacement based on usage and health Object stores like Cleversafe used information-dispersal algorithms to handle slices across nodes — tape needs similar granularity. [Cleversafe: Wikipedia] Cleversafe: Wikipedia D. Multi-Tape Object Distribution D. Multi-Tape Object Distribution Do not pack millions of small objects into a single cartridge. Distribute them across multiple tapes based on: not Object size
Recall probability
Access urgency
Risk profile Object size Recall probability Access urgency Risk profile This is similar to sharding — but for tape. Breaking a dataset into shards across tapes makes individual tape failures less catastrophic. E. Intelligent Tape Management Software E. Intelligent Tape Management Software What is lacking in most environments is metadata intelligence: metadata intelligence Queryable object catalogs decoupled from linear tape structure
Support for erasure coding tracking
Automated tape load/unload scheduling
Predictive recall optimization Queryable object catalogs decoupled from linear tape structure Support for erasure coding tracking Automated tape load/unload scheduling Predictive recall optimization Object preservation software must become tape-aware, not tape-adjacent. Current Tape Market and Preservation Software Landscape Current Tape Market and Preservation Software Landscape A. What’s Available Today A. What’s Available Today LTO Ultrium remains the dominant open tape format with ongoing generational capacity growth and strong ecosystem support. [LTO: Wikipedia]
Major tape library vendors offer high-capacity robotics and drives with strong media life forecasts. [Fujifilm Assets]
Some architectures exist that promote RAIL/parity across tapes (e.g., two-dimensional erasure coding implementations marketed in some products). [Quantum] LTO Ultrium remains the dominant open tape format with ongoing generational capacity growth and strong ecosystem support. [LTO: Wikipedia] LTO: Wikipedia Major tape library vendors offer high-capacity robotics and drives with strong media life forecasts. [Fujifilm Assets] Fujifilm Assets Some architectures exist that promote RAIL/parity across tapes (e.g., two-dimensional erasure coding implementations marketed in some products). [Quantum] Quantum However, most of the industry adoption remains at the “copies + vaulting” level, not erasure codes or cross-tape parity. industry adoption remains at the “copies + vaulting” level B. What Needs to Change B. What Needs to Change Here’s the meat for senior engineering and executive leadership: meat B.1 Tape Protocols Must Evolve B.1 Tape Protocols Must Evolve Tape file systems must incorporate: Native erasure coding support
Distributed metadata
Parallel access paradigms Native erasure coding support Distributed metadata Parallel access paradigms Without this, tape will always be a siloed, brittle medium. B.2 Tape Libraries Must Expose Metadata APIs B.2 Tape Libraries Must Expose Metadata APIs Libraries should: Provide object-level metadata through standardized services
Enable recall planning and pre-fetching
Offer health telemetry beyond simple read/write counts Provide object-level metadata through standardized services Enable recall planning and pre-fetching Offer health telemetry beyond simple read/write counts Until library makers treat metadata as first-class, automation will fail. B.3 Preservation Software Must Treat Tape Like an Object Store B.3 Preservation Software Must Treat Tape Like an Object Store Traditional backup frameworks treat tape as a vault — not as live storage. This mentality must change. Modern preservation is: live storage Immutable
Accessible
Distributed
Self-healing Immutable Accessible Distributed Self-healing Tape must play into all four. C. What Patents and Research Indicate C. What Patents and Research Indicate There are patents tailored to tape protection and cross-media erasure coding: patents tailored to tape protection and cross-media erasure coding US11216196B2 describes applying dual Reed–Solomon erasure codes within magnetic tapes to improve error recovery.
US12099754B2 (assigned to Quantum) covers storing and retrieving objects across multiple media, hinting at a multi-media, potentially multi-tape resilience strategy.
Early work (not tape specific) shows adaptive erasure codes for distributed systems, which could inform tape strategies. [Earlier Patents] US11216196B2 describes applying dual Reed–Solomon erasure codes within magnetic tapes to improve error recovery. US11216196B2 US11216196B2 US12099754B2 (assigned to Quantum) covers storing and retrieving objects across multiple media, hinting at a multi-media, potentially multi-tape resilience strategy. US12099754B2 US12099754B2 Early work (not tape specific) shows adaptive erasure codes for distributed systems, which could inform tape strategies. [Earlier Patents] Earlier Patents In other words: the intellectual groundwork exists, but market adoption lags. intellectual groundwork exists Designing for Tape Failure Tolerance Designing for Tape Failure Tolerance So how do you build a tape ecosystem that can withstand a failure without crippling your preservation program? A. Tape Grouping and Parity Sets A. Tape Grouping and Parity Sets Create tape groups akin to RAID sets: tape groups Parity tapes in every group
Data tapes distribute objects based on hashing
Automated parity rebuild on tape failure Parity tapes in every group Data tapes distribute objects based on hashing Automated parity rebuild on tape failure This model means losing a tape doesn’t automatically lose data; you recover via parity. B. Cross-Tape Redundancy Checking B. Cross-Tape Redundancy Checking Instead of periodic bit scans per cartridge, implement cross-tape consistency checks — compare object references across parity sets and verify content integrity statistically. This is what resilient storage systems do in disk clusters; tape must borrow the idea. per cartridge cross-tape consistency checks verify content integrity statistically C. Object Index with Parity Awareness C. Object Index with Parity Awareness Your catalog must understand: Where parity lives
How to reconstruct
Which tapes are essential vs. redundant Where parity lives How to reconstruct Which tapes are essential vs. redundant Without this, parity is just decoration. D. Tiered Recall Scheduling D. Tiered Recall Scheduling Legacy recall jobs are ad-hoc. Modern systems should: Pre-slot recall batches
Schedule robotic actions with priority
Optimize drive usage Pre-slot recall batches Schedule robotic actions with priority Optimize drive usage This minimizes head and robotic wear and improves predictability. CIO/CTO Takeaways: Risk, Cost, and Strategy CIO/CTO Takeaways: Risk, Cost, and Strategy A. Risk Exposure A. Risk Exposure High object density tapes are single points of failure in a way that disk clusters aren’t. Without parity strategies, you are: single points of failure Facing multi-month restore windows
Exposing your organization to compliance breaches if data is needed sooner Facing multi-month restore windows Exposing your organization to compliance breaches if data is needed sooner Senior leadership must see tape risk beyond “media life” metrics. B. Cost vs. Resilience B. Cost vs. Resilience Yes, erasure coding and RAIT strategies consume some capacity for parity — but they dramatically reduce the operational risk of rebuilds and long recall times. That’s cheaper than a six-month forensic restore or legal penalties. some C. Future Roadmaps C. Future Roadmaps Tape will continue to grow in capacity. But unless architectures evolve with the scale of data and object density, tape will become increasingly brittle. You need: with the scale of data and object density Tape drives with native parity support
Metadata APIs exposed by library vendors
Preservation software designed for distributed, coded storage Tape drives with native parity support Metadata APIs exposed by library vendors Preservation software designed for distributed, coded storage Without this, tape is just a slower, larger silo. Conclusion: Tape Is Not the Enemy — Ignoring Scale Is Conclusion: Tape Is Not the Enemy — Ignoring Scale Is I will say it flatly: tape will remain essential — but only if we stop treating it as a dumb sequential volume and start treating it like a distributed, protected, and codified store. tape will remain essential — but only if we stop treating it as a dumb sequential volume and start treating it like a distributed, protected, and codified store. A single 100 TB tape with hundreds of millions of files is not an asset — it is a bet that you can recall it efficiently and reliably when needed. And right now, that bet is too big for most environments without modern protection strategies. bet too big Tape as liability is not about media — it’s about architecture. Adjust your model. Build redundancy into tapes. Spread objects. Apply parity. And make sure your preservation ecosystem is as resilient as the data you’re trying to save. Tape as liability is not about media — it’s about architecture.

Quant

When “big Tape” Stops Being a Storage Asset and Starts Acting Like Radioactive Data Waste

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

3:2:1 Is Still Necessary. It’s Just No Longer Sufficient.

Could Blockchain and Big Data Come Together To Open Up A New Chapter in Data Integrity?

Data Integrity In Cryptography

Data Integrity Is Vital for The COVID-19 Vaccine Rollout

Data Quality: Its Definitions And How to Improve It

How to protect us from unknown calls?

3:2:1 Is Still Necessary. It’s Just No Longer Sufficient.

Could Blockchain and Big Data Come Together To Open Up A New Chapter in Data Integrity?

Data Integrity In Cryptography

Data Integrity Is Vital for The COVID-19 Vaccine Rollout

Data Quality: Its Definitions And How to Improve It

How to protect us from unknown calls?

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps