Monthly newsletter Weekly news roundup Breaking news notification    

Evidence-based business continuity management

Get free weekly news by e-mailCan the BCM profession learn from ‘evidence-based medicine’? By Patrick Roberts.

Introduction – evidence-based medicine
In 1973 Professor Archie Cochrane published a seminal essay ‘Effectiveness and Efficiency - Random Reflections on Health Services.’ Under the cover of this modest title he proposed a revolution in the medical profession, advocating that treatments should only be administered where thorough scientific proof existed of their efficacy. This approach became known as ‘evidence-based medicine’ and is now championed by a number of ‘Cochrane Centres’ around the world as well as being taught in many leading medical schools.

Arguably, the starting point for the essay was Cochrane’s experience as a Prisoner of War in World War 2 where he found himself as the only medical officer in a camp with over 20,000 PoWs. Under-nourished, lacking in facilities and without modern pharmaceuticals, he expected the camp to be decimated by TB, typhoid and diphtheria. However, in his six months in the camp only four PoWs died: three of those from gunshot wounds. This triggered in the young doctor scepticism about the effectiveness of many conventional treatments and set him on a lifelong search for a more scientific approach to medicine.

Every profession risks overestimating its own effectiveness and I suggest that it is timely, at this early stage in the growth of business continuity management, to review if we can truly justify everything that we do as a profession or if we have already started accumulating the baggage of custom and tradition.

Defining inputs and outputs
The basis of any measure of effectiveness is the identification of appropriate measures for inputs and outputs.

Typical measures for inputs in healthcare are:
* Doctors per 100,000 of the population;
* Hospital bed numbers; and
* Healthcare spending as a proportion of GDP.

Directly analogous measures can easily be defined for business continuity management, such as:
* BC managers per 1000 employees;
* Disaster recovery seats per 100 employees; and
* BCM spending as a proportion of income / profit.

The key measures of output for a healthcare system are mortality and morbidity. Business continuity management is generally promoted in terms of its impact on the survival of organisations i.e. in terms of ‘mortality’ but something analogous to morbidity would actually be a more useful measure. In the same way as the young Cochrane found that the lack of medical care had little impact on mortality, it is unquestionably the case that organisations have survived and prospered for centuries without any formal business continuity plans.

The first major challenge for evidence-based business continuity management is therefore to define an appropriate measure of the output from investment in BCM. The most obvious answer is to measure something like ‘availability of critical systems’ but systems are not an end in themselves and such a metric says nothing about the impact on the business as a whole. As an intermediate step it is preferable therefore to look at a more business-focused measure such as ‘customer satisfaction’. Ultimately though, it should be possible to demonstrate that business continuity management makes a positive impact on the overall performance of the organisation as measured by, for example, profitability or growth.

Measuring effectiveness
The next challenge is to develop a formal process for testing the effectiveness of specific business continuity management activities (e.g. developing plans, investing in disaster recovery facilities and training staff) or BCM spending. Three principal techniques are used in clinical research: these are described below and their applicability to business continuity management is discussed.

Case-control study
A case-control study is retrospective in that it focuses on outcomes. In one of the most famous examples of a case-control study - Doll and Hill’s 1950 study of the impact of smoking on lung cancer - a sample of patients being treated for lung cancer was compared with a similar-sized sample being treated for other diseases. By observing the numbers of smokers and non-smokers in the two samples they were able to calculate that the odds of a smoker getting lung cancer were 9 times that of a non-smoker.

Most existing attempts to justify the effectiveness of business continuity management have been based on a retrospective approach – e.g. the famous ’80 percent of businesses without a business continuity plan fail after a disaster’ quote - but without the rigour of a proper case-control study. The 80 percent figure (even if it is true) is meaningless without knowing how many businesses with business continuity plans also failed. Properly designed case-control studies though would appear to be a very promising avenue for further work: one could analyse similar-sized samples of businesses that failed and businesses that survived a disaster (e.g. Hurricane Katrina) to identify the relative proportion of business continuity plans or disaster recovery provision in each group.

Cohort study
By contrast a cohort study is prospective, that is to say it follows a cohort of subjects over time. Doll and Hill followed their case-control study of smoking with a cohort study. They recruited a cohort of over 30 000 British doctors comprising a mixture of smokers and non-smokers, and follow-up studies of the incidence of lung cancer in both groups were conducted at intervals thereafter. The deliberate selection of a cohort allows for the exclusion of many possible biases (e.g. gender, social class) that may corrupt a case-control study. The drawback of this method is also immediately obvious though: one requires very large sample sizes to have any realistic hope of seeing cases of a rare disease.

Bearing in mind this caveat, conducting a cohort study of business failures could be a very frustrating process. Given the relatively low incidence of business failures, and the fact that only a small proportion of these are caused by operational disruptions of the sort that could reasonably be mitigated by business continuity management, one would need an enormous cohort and a great deal of time to gather any evidence of effectiveness. Returning to the earlier discussion of inputs and outputs, business failure (‘mortality’) is not the only, nor necessarily the most appropriate, measure of output. Useful evidence on the effectiveness of business continuity management activities might be gleaned from comparing some suitable measure(s) of business performance over time across a cohort of businesses some of which engaged in the activity (e.g. conducting annual crisis management exercises) and some of which didn’t.

Randomised clinical trial (RCT)
The randomised clinical trial is another prospective test, often used for drug trials. A cohort is recruited and equal numbers are randomly assigned to either the treatment group or the control group. The RCT thereby seeks to remove as many extraneous effects as possible so as any differential effect between the two groups can be confidently ascribed to the treatment.

Whilst it has been very successful in medical research, it is difficult to see how an RCT would practically be implemented to gather evidence on business continuity management activities. One of the key advantages of the RCT is the ability (e.g. by the use of placebos in drugs trials) to eliminate the psychological effect of a treatment. Whilst it would be interesting to be able to eliminate the psychological effects of a business continuity management programme, there are very few BCM measures that can be effectively implemented without informing people of their existence.

Conclusions
In his essay Cochrane issued a challenge “Which other profession…encourages experimental investigations into the effect of their actions? Which magistrate, judge or headmaster has encouraged randomised clinical trials into their ‘therapeutic’ or ‘deterrent’ actions?” There is much truth in this observation yet these established professions appear to thrive despite the lack of empirical evidence. However, as a very new profession still seeking to establish itself, it is particularly important for business continuity management to be able to provide objective evidence of its usefulness. Hopefully this article has demonstrated that the tools that we need are readily available.

Author: Patrick Roberts is senior consultant Needhams 1834 Ltd
patrick@needhams1834.com www.needhams1834.com

Make a comment

Date: 11th January 2008• Region: UK/World •Type: Article •Topic: BC general
Rate this article or make a comment - click here




Copyright 2008 Portal Publishing LtdPrivacy policyContact usSite mapNavigation help