4 Tips for Prioritizing Metrics

There are three phases to setting up a security metrics program. First, you must collect the data, analyze it and finally report your findings. The Collection phase involves installing sensors (Zeek, Commercial IDS/IPS, vulnerability scanners, etc.). The Analysis phase uses tools such as ServiceNow, Remedy, Crystal Reports, etc. The Reporting phase is the process of creating workbooks (weekly, monthly, yearly) and a set of Operational, Incident and Compliance reports. Your target audiences include your boss, your boss’ boss, IT manager peers, security team, your Board, internal audit, CFO/COO and units involved with regulatory compliance. Matt Tolbert gave a talk in 2007 on Security metrics that resonated with me. Here are some notes I took from his talk for prioritizing security metrics. A good metrics reporting package should include:

1. Operational. Examples of these reports include helpdesk tickets completed, security project status, # of security scans completed and their results, inventory of hardware or software connected to your network. The target audience includes your boss, IT manager peers, and your security team.

2. Incident. These are the number of reported security incidents and their status such as success/failure, financial and reputational impact, “after action reports”, legal status. The target audience is your boss, your boss’ boss and your Board of Directors/Trustees.

3. Compliance. These metrics show how effective your security controls, services and training are in complying with whatever security or data standards your organization has to be in compliance.  The target audience includes your boss, your boss’ boss, internal audit and units involved with regulatory compliance. 

4. Executive. These metrics are similar to the Compliance metric but they also show the value of the security controls, services and training you’ve installed. They should also show areas that need improvement as well as showing progress to meeting the organization’s business goals.

Tolbert suggested 4 benefits from building a security metrics program. These metrics can help:

1. Provide budget and staffing justification for expansion

2. See what risks your organization really faces

3. See what risks your organization will face long and short tem

4. Gauge the effectiveness of your team(s) effectiveness in meeting your security control requirements.

There are a number of good resources on building a security metrics program. Matt Tolbert’s 2007 “Effective Security Metrics” presentation is a great summary.  Andrew Jaquith’s book “Security Metrics: Replacing Fear, Security and Doubt” is another good resource. It’s one of my “bibles” of security metrics. The Educause “Effective Security Metrics: A Guide to Effective Security Metrics” ( https://www.educause.edu/focus-areas-and-initiatives/policy-and-security/cybersecurity-program/resources/information-security-guide/toolkits/effective-security-metrics ) has good high level points to setting up a security metric framework regardless of your industry sector.