Just about every industry is creating massive amounts of digital data. Examples range from media & entertainment (film, broadcast TV/radio content), healthcare (medical images), oil & gas (seismological data), surveillance (video content), bioinformatics (genome sequencing), finance (records), legal (records & discovery), pharmaceutical (research data), engineering (design data)... The list goes on.
This data needs to be stored in a reliable manner. Storage time periods are getting stretched out, in some cases to decades and (I estimate) 100’s of Exabytes of cumulative data between now and 2017 will need to be stored for the long-term.
When retrieved, though not often, the data needs to be accessed immediately. For instance a radiologist needing a medical image or a consumer wanting long-tail video content won’t be satisfied waiting more than a few seconds for it. Something has to be done to address these requirements and it has to be done economically.
Such a storage solution didn't exist (until last week).
Obviously, people are storing data for the long-term using tape, scale-out NAS, cloud storage, or even primary storage. But none of these options provide the features or SLAs to ensure that an object is preserved for decades, available immediately, and without a lot of cost.
EVault, my employer, launched a new cloud storage service last week based on OpenStack Swift that I believe is the first in this category of comprehensively solving long-term storage. It is named Long-Term Storage Service or LTS2 for short.
There are three reasons EVault’s LTS2 proposition will win: 1) unique functionality, SLAs, and the use of open-source software to ensure management and preservation of data for decades, 2) disk based implementation to ensure immediate access, and 3) a unique architecture optimized for long-term storage to provide an eye-popping price of 1.5 cents/ GB/ month. Let's take a deeper look into each of these items.
EVault LTS2 has unique features such as continuous data integrity checking, auditable event tracking, metadata, and usage statistics for management and preservation of data for decades. Data integrity checking, also known as fixity, is the most important feature. If a studio VP asks an archivist whether an archived movie is uncorrupted, with LTS2 the archivist can now confidently say yes. Next is the comprehensive set of SLAs that no other cloud storage provider offers. EVault provides two premium SLAs: data survivorship and portability and two standard SLAs: durability and availability. Data survivorship is the most important one. To put is simply, it is akin to a pre-nuptial. Nobody getting married plans to break-up, and no cloud storage provider introducing a service plans to terminate it. Though it is good business practice to protect against eventualities like the second one. Data survivorship protects the customer in the unlikely case where EVault were to terminate the service. The customer would get a 6 month notice and funds are set aside to return the data back to the customer or another cloud storage provider using physical media. Customers love this SLA since it protects them against a Nirvanix like situation (Nirvanix was a cloud storage provider that went out of business recently and left their customers in a lurch). Finally LTS2 uses OpenStack Swift. Open-source software has been shown to out survive people and companies more effectively than proprietary software. The fact is that Swift based public clouds are offered by other vendors such as RackSpace, IBM, HP, Internap etc. A number of companies also provide private clouds based on Swift.
Next is the use of disks. This ensures that data is always available, and can be retrieved in seconds as opposed to hours or days. The access can also be granular and random. This combination enables new business models such as long-tail video on demand, video micro-clips, collaboration using large medical images etc. Clearly, innovating on disk technologies with its parent Seagate is a significant advantage for EVault. See my previous blog on Seagate Kinetic.
Finally LTS2 is priced very aggressively at a flat 1.5 cents/ GB/ month. Unlike other services, there are no retrieval fees, there is a very generous amount of data that can be transferred out for free every month (5%), and two SLAs are included for free. No other cloud storage service can match this. In fact some services that superficially look less expensive than LTS2 actually turn out to be significantly more expensive due to retrieval, transfer, and request costs.
I am obviously a bit biased :-) but I'm truly excited about EVault’s LTS2 because I believe we are the first in the long-term storage category. As such EVault will dominate this category. Try it out!