Incident Response

Chapter 4. Mission and Capabilities

When you have a clear understanding of your clients and what their problems are, it is time to start developing some solutions. At the top of the list should be planning the team’s core services. As we’ve previously discussed, there is a wide range of possible services a team can provide to its customers. The actual list of services will obviously vary somewhat by organization, but the list should be carefully thought through and decided on, although it is bound to be revised over time. Also understand that the list will probably be driven not only by emergency services, but by services that can be performed during times of noncrisis (however rare). These nonemergency services can be used to justify a team’s existence by doing things like awareness training to raise the corporation’s overall level of security.

One effective way of coming up with a reasonable list of core services is to start by defining only the emergency-driven services. These are likely to include emergency hotline support, hands-on crisis response technical services, incident investigation support, advisory distribution, and technical support for installing vendor-supplied security patches. Once you have that list and have established a good understanding of what each service entails, make a reasonable estimate of the resources required to perform these services and the percentage of staff time they are likely to take. From that, fill in the gaps by planning nonemergency services like awareness training, technical security seminars, and any other services the corporation can benefit from.

Also, before jumping in and defining the services, consider visiting some of the key business unit managers and asking them what sort of security services they perceive a need for. It might be that they don’t see a need for any, or few, based on a lack of understanding of what might be available, so this is also an opportunity to educate them somewhat. At least for most corporate incident response teams, the business managers are the end customer of the service; showing that you really will provide the services they need most is certainly time well spent. This is especially true if the business units are going to provide some of the direct or indirect funding for the team’s services.

The time you spend defining the services now is also valuable. This list of services will almost certainly become the “menu” of services the customers see. Any restaurant owner can tell you how important it is to have a menu that consists of services your customers will actually want.

Although the main reason for documenting the core services now is so that you can plan the team effectively, a positive side effect is that your customers are likely to have a better understanding of what you are offering. If they understand the service, they are more likely to make use of it.

Roles and Responsibilities

Right now, you’re probably tempted to put the book down and scramble to assemble your elite squad of cyberparamedics. But whoa, partners! There is more than simply figuring out what your core services are! Special times (such as incidents) require special people.

Staff selection should be driven by the list of core services your team will provide. Will you support all of the computing platforms, operating systems, applications, and services in use at your organization, or only a select few? Once you identify those core services, you can address the more engaging questions about how many people you will need on the team, their required technical skills, and what experience levels you will need. Typical job positions include a team manager, a senior technical leader, assorted technical and nontechnical engineers, analysts, and assistants.

Before even thinking about how your new team will operate, you need to know the whos, whats, and hows of the team. First you will start thinking about who you want to include on your team based on qualifications and position. But that’s not enough. You then have to know what job positions (role) each person will function on the team, and also how well they can function under pressure as both an individual and member of an elite technical team.

To many managers, two of the most dreaded words in Corporate America are “job description” and "performance evaluation.” Still, documenting each of the team’s staffing positions is a necessary process. Your job descriptions should include a list of what each position (team leader, senior technical lead, technical staff, and so forth) on your team is expected to do along with the necessary skills, experience levels, and educational backgrounds necessary for that position. The list of duties should be as detailed as possible, with the inevitable “other duties as assigned” included for good measure. The good news is that this information can be used for each employee’s performance criteria, especially if the job description is sufficiently detailed.

Closely related to the job descriptions are the roles and responsibilities. During a crisis, it is all too easy for people -- even senior staff -- to become unfocused or unglued due to the strain and stress of the situation. While working an incident, there are always more tasks than available people to perform them. This translates to high stress that can result in this loss of focus. Insuring each member of the team knows and understands their roles and responsibilities helps maintain a clear focus on the task at hand. This is especially true for any hired guns who are used on the incident, whether they are matrixed staff pulled in from another business unit or contracted staff from a commercial team. Everyone needs to have as clear an understanding as possible of what he should be doing.

Common incident response roles and responsibilities include the following:

Team Manager

The Manager is responsible for the overall administrative and personnel management of the team.

Team Leader/Incident Coordinator

This person is responsible for leading a particular incident response operation or effort. The Team Leader makes the operational decisions on how to respond to the incident he is called out for. The Team Leader is accountable to the Team Manager for the progress and resolution of the incidents he is working on.

Senior Information Protection Engineer

The Senior Engineers are experienced engineers who provide the senior technical effort for the project. During an incident, the Senior Engineers report directly to the Team Leader.

Information Protection Analyst

The Analysts provide the core labor for the incident operation, at the direction of the Team Leader and the Senior Engineer. They report to the Senior Engineer.

Equipment Custodian

Reporting to the Team Manager, the Equipment Custodian is responsible for providing the team with all the equipment needed to conduct operations, software as well as hardware.

Deployment Logistics Support Officer

Reporting to the Team Manager, the Deployment Logistics Support Officer provides the team with all the logistics and administrative support necessary for handling an incident. This is likely to include travel arrangements, clearing calendars of other appointments, and deflecting other nonemergency work tasks.

This list illustrates typical roles and responsibilities. We should point out that many of these “positions” are temporary in the sense that they only exist during an actual incident response operation. Further, the people filling the roles and responsibilities might shift from one incident to another.

But what if you’re the director of information technology for a public school or nonprofit association that has extremely limited resources available for technology in general, let alone for computer security or incident response? Do you simply throw up your hands in despair and run for shelter during an incident? Of course not.

Improvise, Adapt, Overcome

Although this book assumes that you have appropriate resources, we have firsthand experience that this is rarely the case. As such, the adage “Improvise, Adapt, Overcome” comes to mind. We’ve worked in offices with only one system administrator for over a hundred users of various skill levels. As good system administrators, we knew our users and their skill levels, and were able to draft some of the power users into service to assist in cleaning up a virus incident or even upgrading operating systems! Many times such power users are happy to assist with what they know and be recognized as a power user in the organization.

For the beleaguered system administrator dealing with an incident, this can be a very useful cadre of temporary staff to help navigate the wickets of the day. However, take care to monitor what these volunteers do -- ensure they do what they are told to do (e.g., they don’t get creative and break something else), that they understand they may come into contact with sensitive information, and how to deal with such findings. As the system administrator, you need to make yourself available during times of crisis to assist or guide your surrogate staff if they encounter problems.

Each team member and person working an incident must fully understand what is expected of him. Roles need to be clear and understood by everyone, and the senior staff should be pedantic about supervising each person’s responsibilities. Confusion leads to mistakes, and mistakes will not help resolve an incident nor will they help in a criminal prosecution of your attacker.