Preparing a power failure response strategy
By Dr. Jim Kennedy.
According to a recent Forrester Research survey nearly half of the declared disasters reported over a one-year period were due to power failures. To ensure that this type of event does not escalate into a disaster that results in business disruption and lost revenue for your organization you need to be prepared. And this requires a power failure response strategy as part of a wider business continuity plan.
This article explains how to develop such a power failure response strategy.
Determine what you have
An up-to-date electrical system schematic or wiring diagram will clearly identify how the main components of the electrical system are connected. It should include any special systems, such as uninterruptible power supplies or power protection devices. Specifics regarding equipment voltage and current ratings should be identified and documented. It is also important to identify each major component with a unique name describing its location and important power ratings.
In addition to identifying the equipment on the drawings, it is equally important to physically match the system or component to the drawings by affixing a permanent label to the actual equipment for identification purposes. This, along with a flashlight, facilitates finding the equipment quickly in the case of an emergency.
Determine what your organization’s mission critical requirements are for emergency power
Once the electrical power distribution system is documented and understood, the next step is to identify business critical loads that will require emergency power in the event of a power failure. At this point it is important to note that uninterruptible power supplies and emergency generators may not normally provide for all of an organization’s electrical needs. So, decisions will need to be made to determine what equipment in the operation in time of a disaster or incident absolutely need to have power maintained during a power outage.
As I indicated earlier, this is usually done as part of an overall business impact analysis, where business critical systems and devices within the organization are identified. These are systems for which special considerations need to be made in case of any type of failure — of which power failures are only one example.
After critical systems have been identified their sources of power within the facilities need to be identified on the electrical drawings to make sure it is understood that they are critical to the operation (i.e., critical computing systems, LAN/WAN network devices, PBXs, production and/or security equipment). If the electrical distribution system has been properly designed, all business equipment should be connected to common circuits and separated from non-critical equipment. This makes it much easier to isolate mission critical circuits and to properly connect them to emergency power sources during an emergency. It is also important to keep the documentation up-to-date for the equipment contained on these circuits, so that the electrical load (current required to keep the equipment running) is properly identified and emergency power generation equipment can be properly sized.
Determine what type of emergency power your organization should use
Portable electric generators can also be used. They can be installed after the power has failed and before the UPSs would shut down. If a portable generating unit is to be used, it is absolutely critical that the electrical load of each emergency circuit (measured in amperes) be known so that the generator(s) can be correctly sized (be large enough to supply the rated current of all devices to be powered during the power failure). The wattage and voltage ratings for each generator should be defined prior to the outage and procedures documented, so that when acquiring them, the load does not have to be calculated under adverse conditions: which usually leads to error and further disaster. If it is decided that the use of portable generation is the best alternative, it is prudent to arrange a pre-emergency contract to rent or lease the generators.
If a pre-rental or lease agreement is undertaken, be sure that the contract specifies delivery time and maintenance if a provided system/component fails to operate correctly. It is also important to understand in what priority order you your generators will be delivered especially in case of a widespread emergency outage. However, even with a pre-contracted agreement in place, there is no real guarantee that the generators will be available if the power outage is widespread.
It may be in your organization’s best interest to acquire portable generators ahead of time and store them for a time when they are needed. If you choose this course, it is important to establish and implement proper maintenance procedures (e.g., running the generators periodically and changing the gas and oil as recommended by the manufacturer) to make sure that the generators function when needed.
Making the generators ready for the next outage
All of the necessary connection equipment (e.g., cables and connectors, etc.) should be stored near the place where the generators will be used. In addition, fuel and operational supplies need to be planned for as generators will consume large supplies of gasoline and oil if they need to run for many hours. Emergency generator procedures need to be well documented beforehand and attached to the generator or the transfer switch. This ensures that the generators will be connected properly and will not damage any wiring or critical equipment.
If the connection is very complex then part of the emergency response procedures should include an on-staff or contracted electrician or experienced technician to perform the connection of the generators.
Ensure the generators are functional when needed
The generators should be tested under actual circumstances and full load, if possible, but, if not, then a simple functional test will have to suffice. This functional test should include starting and running each generator for a minimum of thirty minutes at least every month or minimally, once each quarter. Also make sure that each generator’s fuel tank is full upon completion of each test and that there are documented procedures to obtain fuel during an emergency (perhaps the local gas station may not be the best source). Manufacturers of generating equipment typically recommend that fuel be replaced that is more than one year old.
Formal process and procedures are critical to success
During a crisis, people react differently than when there is no stress, so the procedures should be clearly written. One method to make the procedures easier to follow would be to use step-by-step check lists or flow-charts. Having these procedures available at each generator site to connect each portable generator will save a great deal of time during an actual crisis event. It is also important to note that the procedures should include both how to connect the generator and how to disconnect the generator when power has been restored.
Running generator installation tests periodically can help make sure everyone is comfortable and prepared for the time when the drill may be replaced by a real power outage situation.
About the Author
In addition, Dr. Kennedy has authored numerous information/cyber security and business continuity articles for US and international publications, has presented at numerous national conferences and seminars, and has taught courses in these disciplines at the university level. He has also developed many information security management and assurance programs for pharmaceutical, educational, financial, petrochemical, energy, transportation, and telecommunications organizations world-wide.
The author can be reached at: Recovery-Solutions@xcellnt.com
•Date: 11th November 2011 • Region: US/World •Type: Article • Topic: BC plan development
To submit news stories to Continuity Central, e-mail the editor.
Want an RSS newsfeed for your website? Click here