I interact with all elements of our IT culture. I consult with end users and vendors, and I interact with reporters and analysts. Lately, what I've heard from reporters and analysts sounds a lot like this:
Blah blah blah BIG DATA blah blah blah CLOUD blah blah blah CLOUD blah blah blah BIG DATA.
Oh, sure, sometimes they throw in words such as "virtualization" and "hybrid" and "e-discovery," but the overarching message doesn't vary much.
Although I think the overall intersection between cloud storage and Big Data is an interesting topic to explore, some of the proposed approaches are more practical than others.
Not Hot: Cloud Storage as "The Answer" to the Big Data Challenge
To put us all on the same page, let's start with a definition of "Big Data." Big Data refers to the unprecedented growth of unstructured data within organizations. We're being overrun by email, video files, audio files, and just about every type of file you can think of. Moreover we're being forced to retain this unstructured information for increasingly longer periods of time (thank you, regulations!).
Organizations are therefore forced to buy more and more storage (which, by the way, will be challenging and expensive given the flooding in Thailand, where a lot of hard disk drives are manufactured). An oft-proposed intersection with cloud storage is: Let's move all the data we can out of our data centers and into the cloud so we don't have to worry about all the immediate consequences of Big Data.
I have no problem with the concept—it's the execution that bothers me. On the technology front, I worry about having sufficient bandwidth (with low latency) to actually move information to and from the cloud in a timely fashion. It's one thing to back up the My Documents folder on a laptop to the cloud; it's an entirely different beast to back up data center files. Furthermore, even if we can move the information to the cloud, can we get it back in a timely fashion if we need to recover? I would assert that you cannot restore all of your data center storage—probably not even most of it—from the cloud in a reasonable time frame.
Finally, there are the annoying issues of security and responsibility. Are you sure your data is safe up there in those fluffy clouds? What real assurances do you have from your cloud vendor that it will never lose your data?
Color me skeptical, but these concerns definitely lead me to a "not hot" conclusion on the use of cloud storage as the "silver bullet" solution for the Big Data challenge.
Hot: Cloud Storage Helping with Big Data Challenges
If you can get past the idea of cloud storage as "the solution" for Big Data, you can start to see how storage in the cloud can actually be helpful. As I noted, it's an interesting option for end-user backup (at least for end users who have high-bandwidth connections), and it can absolutely play a role in supplementing your current data center storage strategy.
The key word there is "supplement." Cloud-based storage doesn't replace tape storage. It doesn't replace near-line disk storage. It's simply another tool in the toolkit. You may want to, for example, move archived data to both cloud storage and tape storage selectively. For instance, full system backups are better suited to tape, but targeted file backups may reasonably go to both tape and cloud storage.
Why use both tape drives and cloud storage? As I noted, the big problem with cloud storage is time-to-restore from a disaster. Tape may not be the fastest thing in the world; however, it's much faster than restoring entire servers from the cloud over typical Internet connections. On the other hand, sequentially searching through a tape for a specific file (or set of files) will be much slower than restoring from the cloud.
I don't know about you, but I like lots of tools in my IT toolbox. Cloud storage may not be a universal tool for all your storage challenges, but it can be the right tool—the hot tool—for specific Big Data problems.
Sean Chandler is a computer and network consultant with more than 30 years of field experience. Astro, a border collie with more than 40 dog years of data processing experience, provides technical support to his master, Sean.
Astro's Pick of the Litter
As embarrassing as this is to admit, my master has purchased yet another Android tablet. But his recent purchase is a little different from the others. He bought a Kindle Fire. It's true that the Kindle Fire is based on the Android operating system, but the Kindle Fire is closely tied to the Amazon ecosystem. For example, when you configure a Kindle, you use your Amazon sign-in information and not your Google sign-in information. My master—who has always been a fan of the Kindle experience—loves this association and appreciates how the Kindle Fire further leverages Amazon's investments in music, video rental, and video streaming. And if he gets sick of the Amazon ecosystem, he can still load other Android apps to support other media sources.
—Astro