There was an interesting short article in Computer World on Monday, August 27 that described a trend in which outsourcing customers are bringing disaster recovery back in-house. Having witnessed this debate on an ongoing basis several times, I read this article with a combination of cynicism and intrigue.

The article described four reasons for this trend: tightening timelines at a cost lower than offered by vendors; bringing recovery centers closer to customers’ data centers; inflexible, long term contracts; and inflexible testing options and environments. While there is certainly truth in each of these, I believe that there are two unmentioned reasons, both more compelling than those above, that were overlooked by this article.

First among these is that companies have often bought disaster recovery on an al la carte basis. One system is outsourced, along with the DR requirements for that system, followed by another system – maybe to a different vendor. A company can wind up outsourcing multiple processes under multiple agreements to multiple vendors – with each deal providing DR services separately.

Like going to a retail store and purchasing an extended warranty on specific, individual products, buying DR on an a la carte basis is a very expensive way to purchase these kinds of services. And like a smart consumer who makes the risk-reward calculation and then declines such extended warranties, companies will, when possible, say “no” to such extra costs.

Second and more cynically, is that an outsourced DR process is very difficult to de-fund. A vendor knows that it must meet its service levels – and often there is enhanced liability for failing to meet DR requirements. Doing so, and accepting that risk, is a very expensive proposition (and also a source for some windfall profits). Some vendors even try, contractually, to limit a company’s ability to provide thier own DR services by limiting that company’s ability to have extra, operational instances of software and systems. On the other hand, an in-house DR process can be cut – often drastically – by a company which determines that the risk-reward calculus simply does not warrant the investment required by a true, company-wide DR investment.

Companies cut costs by only providing partial DR for their own systems. Major systems are often tested separately, allowing companies to boast quick recovery of disasters on such systems, but hiding from themselves, their management and their shareholders the inability to recover from a company-wide, multi-system disaster. They cannot do this with outsourced services – as the contract calls for all systems to be available within certain timeframes – and the true liability risk is borne by the vendors.

It is important to remember that DR is, in fact, an extra cost related only to the expense-side of a business. No one (except for vendors providing DR) makes a cent on any dollar spent on DR. Companies looking to reduce costs have been ruthless about cutting the expense-side (rather than the revenue side) of their businesses – and, frankly, in-house DR is among the easiest of these targets. It is simply a meta-expense. Even after the tragedies of 9/11 and Hurricane Katrina, no major company has failed (or even suffered substantially) due to a failure in its DR procedures.

Of course, there are ways to solve this conundrum. Foremost of these is to move business continuity issues into a separate budget area controlled by the company’s risk manager, not its CIO. CIO’s are constantly faced with tough budget choices and, because of the “black box” nature of many of their projects, in-house DR is often the victim of such budget choices (can’t do this with outsourced DR!). Treating DR like insurance, on the other hand, would allow for more rational, transparent behavior.

Second, eliminate the a la carte DR approach. While it often seems easier and cheaper up front, the cost over time is much higher than a centralized, organized and coordinated DR infrastructure. Doing this may require extra staff and expertise to manage, but it can lead to extraordinary savings and extra security – and higher quality DR.

Third, look for ways to have your vendors work together on DR. If your company has several outsourcing vendors providing similar services (say, BPO, ITO, call centers), develop in your contract requirements that would have each vendor act as DR providers for your other vendors. Of course, many vendors will squawk about “proprietary processes” and will be reluctant to agree to work with other vendors, but at the end of the day, an honest look at what is (and is not) the secret sauce of a company or vendor can reduce this concern dramatically.

Finally, be honest about the costs of true DR – and what a company is truly willing to spend and provide internally. Critical systems – particularly in certain industries – may require hot sites, mirrored storage, extra staff, etc. Other systems may be able to tolerate disruptions of 24 hours or longer. Match your DR standards to your true requirements – and pay only for what you need.

Disaster recovery is an important responsibility of every company. Like commercial general liability insurance, no company should be without it. However, unlike CGL, where needs are well understood by management and a rational market has developed, disaster recovery remains another IT “black-box,” and remains subject to the whims of the ever-more-ruthless budget cycle. Maybe it is time to move disaster recovery out of the shadows and treat it, like insurance, as a truly necessary expense.