Protecting the Enterprise: How a Well-Designed Security Analytics Programme Can Help

Determining metrics and getting data

There are many tools and technologies used by today’s information security teams to manage and protect their environments. Fortunately, data can be leveraged from all of these tools for analytics purposes, as long as the tool allows access to the underlying database, the exporting of data, or a connection to the data via API. Cloud technologies such as Microsoft Azure, Amazon AWS and Google Cloud Platform are all commonly used by companies to consolidate, transform and model data. Key performance indicators (KPIs) and key risk indicators (KRIs) are then visualised through reporting solutions such as Microsoft Power BI, Tableau and Qlik.

Deciding what security metrics to collect is typically done in workshop sessions with input from the various stakeholders of the process being analysed: executives, business leaders, operational/ IT managers, and others. The input from these sessions then helps IT and the data stewards to prioritise metrics, key data sources, key data points, and data segmentation options.

Each agreed-upon metric will need the following information:

• Data points required to support the metric
• A data source to retrieve the data
• Segmentation options for the data, such as business unit, and time frequency, such as day, week or month

For some metrics the required data sources, data points and segmentation options may not be captured at the initial discussion but the metric can still be prioritised so that the feasibility of obtaining data for it can be determined in future iterations.

An approach to developing security metrics

The goal-question-indicator-metric (GQIM) process is a generally accepted framework developed by Carnegie Mellon University that Protiviti leverages when assisting clients with metrics development. GQIM has proven to be a well-defined and repeatable process to derive meaningful metrics that directly support the achievement of business objectives. This method allows IS professionals to select metrics by demonstrating each metric’s business value in informing business decisions and influencing action and to retire metrics when business objectives change.

Applying the GQIM method to vulnerability management analytics — an example

One area where organisations most often develop security analytics is vulnerability management. Vulnerability data is one of the most valuable inputs when determining the security posture of an organisation and is well suited to be leveraged as telemetry for an organisation.

Begin by understanding what business objectives establish the need for vulnerability management metrics and the goals for each business objective. Once those steps are complete, it will become easier to formulate the questions you want to answer. The questions should be pertinent to specific problems that a vulnerability management function is responsible for managing. If you’re not sure what the problems are in your programme, a quick selfcheck or a third-party assessment of your programme’s current state can help to identify any critical problem areas.

The table below provides examples of possible questions and metrics, based on the maturity of the organisation’s vulnerability management programme.

Once the problem areas have been identified and ranked in importance, establish a list of indicators that can be used to determine if the questions relating to the problems have been answered or not. Lastly, create a backlog of metrics that provide insights into whether the problem state is degrading, improving, or has been eliminated.

Metrics based upon vulnerability management data are essentially “point in time” metrics, so it is important to create a telemetry system by ensuring the ongoing, frequent and automated availability of data points to provide near-real-time information to IT and the executive team. It is also important to understand that your first metrics are not going to be your last metrics as new problems may develop as processes that vulnerability management monitors change. Share your metrics with working groups at key points along the process chain and with senior stakeholders to allow outside voices to help govern the overall process and provide a vehicle for continuous improvement of the programme’s telemetry and metrics system.

How to get a security analytics programme off the ground

The first step to the successful establishment of security analytics is alignment. Identify a project sponsor and key stakeholders who can work together to define project objectives, expectations and desired outputs. The project needs to align with business unit and overall company objectives.

Next, define technical requirements by reviewing the existing environment available for sourcing data and developing reports. Conduct an inventory of the appropriate applications, databases and tools for sourcing the data and request access and/or software licenses if needed. A target-state architecture diagram is typically developed to ensure there is agreement on the technical infrastructure that will be utilised or implemented.

Solicit stakeholder input to identify key metrics as well as to validate and prioritise data sources and their key data elements. You may want to develop wireframes using a subset of data from agreed-upon sources as a way to gather feedback from stakeholders and tailor the end product.

Once you’ve reached agreement on a prioritised list of metrics and their data sources, you can commence development. Data pipeline development entails the development and automation of data ingestion procedures to move data from source systems through the target architecture. The wireframes can be used as the basis for the visualisation component of the build. Once data is flowing through from source to the end report, it is important to validate the data, gather additional feedback, and iterate on the end product used by the stakeholders.

Finally, continue to solicit support from the business to ensure the success of your security analytics programme, meaning the programme output continues to meet business needs and evolves as those needs change.

Demonstrating value and getting traction

Before investing significant resources in a large-scale security analytics programme, many organisations develop a proof-of-concept to help demonstrate the value of a security analytics project and to get stakeholder support. Typically, this is done using a subset of data to develop sample visuals and provide stakeholders insight into what is possible and how it would be useful to the organisation. This approach requires a smaller upfront investment because the technical infrastructure component is removed from the build.

Alternatively, the information security team may take the approach of selecting a specific area within the information security realm — such as vulnerability management — to pilot an analytics telemetry project with reports using live data for near-real-time analysis. While the investment with this approach is larger, the end product delivers more tangible value.

Finally, when ready, organisations will want to bring multiple areas within information security in the security analytics programme. Implementing analytics across multiple realms is the best way for information security organisations to have an ongoing, 360 view of the cyber threats facing the organisation and to coordinate action to address them.