Magnetic tape storage will not disappear any time soon but, increasingly, it is being driven to the edge of system storage, into being used primarily for deep archive. There is good reason—and it involves a combination of virtual tape library (VTL) systems that utilise low cost SATA disks and de-duplication.
It is many decades since disks (even with very small capacities) opened up an on-line market by providing random access; this displaced tape in the primary storage tier. However, since then, even though disk drives have massively increased in capacity and speed, tape technology has largely kept pace and maintained a considerable price differential to deter detractors from replacing it for sequential backup and recovery operations.
However, the massive and inexorable increase in storage and shrinking backup windows have together created a bigger management headache when tape is used, for instance in locating and loading the correct tapes when recovery is needed. When tapes are physically transported off site the risks and problems are multiplied; there is the danger of damage or theft in transit, plus potentially huge delays in retrieving and returning the correct tapes through busy traffic even before recovery can begin. Not very 21st century, that.
De-duplication appliances for backup started really gaining popularity in the mid-market about three years back. Inserting a NAS de-dupe appliance, transparently (and this is important), into the backup process means an average 20:1 (95%) saving on the output data storage need with little performance overhead even over a straight disk-to-disk backup. This alone dramatically altered the economics and ROI versus tape; a disk backup (de-duped) suddenly cost no more than a tape backup (not de-dupable)—as well as being far faster (especially for recovery), more reliable and simpler to manage.
A further bonus is that off-site replication in de-duped form can be achieved very fast and reliably over WAN, at very low line cost, so can remove the risks and time normally incurred if transporting backup tapes between locations; this has also opened up a low-cost storage disaster recovery (DR) capability. So the ROI case has become very straightforward for most mid-market companies. We have therefore seen de-dupe appliance vendors Data Domain, and latterly ExaGrid, in particular doing very nicely thank you in the mid-market—with tape being squeezed out altogether in many of their accounts.
On the other hand, large enterprises have a huge legacy investment in tape for backup and archive storage. Enter VTL. The backup application thinks it is writing to tape but VTL redirects it to disk; so initial backup is faster while the data can be retained ‘near-line' for much longer—allowing rapid on-line access as well as fast recovery. The transfer to the physical tape library drives then happens as a more leisurely background task. Despite this, making an ROI case with VTL alone may not be easy.
However, VTL that incorporates de-dupe is a different matter. This combination can use only disk for backup and add remote replication—leaving only the archive and remote backup tape libraries untouched. Now the major hardware and backup software vendors are offering VTL/de-dupe solutions alongside, or in competition with, specialists such as Sepaton and Data Domain. Even z/OS mainframes are now catered for with an offering from Data Domain partner Shoden Data Systems.
Despite this, VTL with de-dupe is actually a stop-gap solution (albeit for the medium term). It avoids an enterprise being faced with disruptive system changes in order to replace tape with disk for backup and gain the storage capacity savings; but it also carries an overhead in formatting the output to one of the tape formats—which is especially inefficient when there is no tape at all at the back-end of the process (as VTL solutions allow as an option).
So tape will survive for now, especially for off-line long-term deep archive storage; but even here management headaches need addressing, including the changing tape formats and drives causing old data to need refreshing by being copied to newer format tapes—or maybe disk—on replacement equipment to remain readable. Moreover, de-dupe can be applied to the archive and typically achieve a 4:1(80%) data reduction as long as tape is not used.
I would be very surprised if the tape storage vendors had a viable answer this time around. So, while it may be a slow death, the end for tape for backup and storage is coming nearer with the demise of tape archive only just beyond the horizon.