Disaster Recovery: The Four Pieces

I don’t care where you’re from, we can all relate to this one.  It doesn’t matter if you’re from the United States, Brazil, Russia, China, or wherever.  Someplace, somewhere,  we were chowing down on something as a child and taking huge bites and just cramming our mouths full, and our mother gave us that look that only a mother can give, and said “Smaller bites.  And don’t stuff your mouth.  Do you want to choke to death?”

Well, that is a bit like Disaster Recovery.  It’s such a big subject, people dive into it and they soon get choked by it.  It’s a huge and overwhelming topic.  Well, let’s break out a knife and cut it up into smaller pieces.

Disaster Recovery is broken down into four pieces, and oddly enough, you’ve probably done them all, you just didn’t know it.  The four pieces are:

  • MITIGATION – You look at the threats, and you try to make them either go away, or minimize their impact.  This can be a serious undertaking considering, that human race has yet to figure out how to divert a tornado or head off an earthquake.  But there are some things you can do something about.  Since power outages are in the top things that can impact your company, it’s actually something you can easily mitigate against.  Viruses and hackers are also something you can mitigate against.  And so is data lost (we call these backups).  For everything else, there’s planning, and most of that centers around protecting your data, and deciding what it’s going to take to get back up and running.
  •  PLANNING – You can’t just have any plan, but rather a plan, that works for you.  There will actually be two Disaster Recovery plans in the company.  One is the big umbrella plan that fits the whole company and has zero or very little to do with IT (though we need to have a place at that planning table).  The IT plan on the other hand needs to look at the mitigation pieces we’ve implemented and how we maintain them.  We also need to have written out how to do system restores.  In this, I like simple, 4th grade language with lots and lots of pictures.  Murphy’s Law dictates that you’ll be doing this at 3 AM and you don’t need to be trying to figure out what you had written six months before.

An often overlooked part of planning is the need to exercise the plan.  You can have the most beautiful plan on paper possible, but if it doesn’t work, what’s the point.  An exercise is designed to find the flaws as well as to get you familiar with how things are done.  More on exercises later.

Your plan needs to provide for updating.  As our technology changes, or the threat landscape changes, or both, we need to adapt the plan to reflect that.

  • RESPONSE – It’s 3 AM in the morning, the phone rings and it’s your boss telling you that your datacenter is now a smoking hole in the ground.  Recovery is about so much more, than just getting things up and going.  Something most people don’t remember is that many communications have to take place to pull this off.  How do we do that?  There’s also a lot of tracking of hours, and expenses that need to be reported.  What kind of expenses?  Well, that new server, obviously.  But how about Pizza to feed everyone, or mileage running down to the local electronics store for cable.  Response needs to have that built in to it as part of the planning process.
  •  RECOVERY – Somewhere in the response phase, recovery begins.  It can be as simple as the CEO announcing the location for the new building, or you pull out the DVDs to rebuild a server.  The important thing about recovery is once it’s done, you need to sit down and look at it and ask what worked well, what worked OK, and what demands more effort.  And then you go back and start looking at what happened, and you start planning on making sure it doesn’t happen again or the impact is minimized.

Strange, sounds like we’re back at Mitigation.  And you should be.  Disaster Recovery planning is a never ending process, to which you need to pay close attention.