How do you test a new emergency notification system?

Tell, Teach, Involve

Training and Learning Techniques for Emergency Notification and Incident Notification Personnel

“Tell me and I forget, teach me, and I may remember, involve me and I learn.”

― Benjamin Franklin

Recently, I received the following question:

I have looked at a variety of ways to test an installed system over time, but what can you do for newly installed or updated system testing?

Acquiring a new emergency notification system requires a thorough test plan of the system’s features and your team’s ability to implement, use, and monitor those features based on your needs. Message content, contact database lists, notification requirements, policies relating to the use of multiple devices, and confirmation guidelines have to be defined and tested.

Case Study

To explain how to test a new emergency notification system, I will use a case study, “Developing and Testing an Emergency Notification System for a County Emergency Management Agency,” by David Edwards and Danny Peterson from Arizona State University and Holly Cuthbertson, California Department of Public Health. It was published by the Journal of Homeland Security and Emergency Management in 2011.

In the case study, the authors reviewed the implementation and testing of an emergency notification system in a county-wide, multi-agency functional exercise. The focus was on notifications primarily to internal staff or agencies. There are two quick points we should think about in the discussion.

1. What systems must be tested? Understand that you need to test both the system’s capabilities and your team’s ability to rapidly and safely use those capabilities. For example, during the Boston Marathon bombings, text messaging was the most reliable way to get the message out. The difference between, “We can deliver text messages,” and the actual pre-testing provided greater confidence and results. Understanding the difference between SMS versus SMTP messaging capabilities was crucial.

As we noted in an earlier blog entry, the testing can range from tabletop exercises for the staff to full-scale operational exercises. In the case study, the authors point out that full-scale exercises are more costly but are needed to ensure that the required functionality and usability are present. Your ENS provider should be able to help you plan the necessary testing and provide an estimate of the cost.

“Use of an ENS in exercises involving unified command conditions can enhance coordination and cooperation among organizations and prepare staff for real emergencies. Through regular testing of an ENS under highly realistic conditions, strengths and weaknesses of the system and its implementation can be thoroughly explored and identified. Valuable lessons can be learned.” (Edwards, Cuthbertson, and Peterson, 2011).

2. Does the system meet your requirements? The tasks and tests should cover the requirements that were used to define your site’s system requirements from the vendor. The critical aspects need to be tested and evaluated ― it is very risky to wait until an emergency occurs to discover one required feature does not function correctly.

In the case study, the tasks included:

a. Documentation on the completed system implementation

b. Review of service-level agreements for access to the system and notification delivery

c. Written procedures for notifications and message mapping

d. Accurate and reliable loading of contacts in the groups required for scenario-based testing

e. Training of employees in the use of the system

f. Awareness campaign for testing stakeholders and recipients on what to expect from notification tests – and what is expected of them

g. Initial small-scale tests to gather review from site stakeholders and analyze system result reports

h. Proposed system modifications based on after action reporting

i. Full-scale operation exercise reaching the full target recipients

j. After action reporting based on recipient reaction and analyzed system result reports

Test your emergency notification system to understand its features and ensure your team can send messages throughout the lifecycle of a crisis.