Information security indicators

In information technology, benchmarking of computer security requires measurements for comparing both different IT systems and single IT systems in dedicated situations. The technical approach is a pre-defined catalog of security events (security incident and vulnerability) together with corresponding formula for the calculation of security indicators that are accepted and comprehensive.

Within the recent years, many companies and organizations have emphasized the importance of security indicators (ISIs). These indicators are measurable signals that can help assess, monitor, and improve the security of an organization. ISIs can translate intricate security processes, breaking them down so they can be observed easily so decision-makers can understand if their defenses are working well, if there are risks, and where they can improve.

Definition and Purpose

An industry definition states that security indicators are "values based on metrics obtained by comparing logically related attributes about the behavior of an activity, process, or control within a specified time" (Gartner). This includes the number of unpatched systems, average time to detect breaches, percentages of successful breaches, attempted attacks, and more. Indicators like these are essential because without this data and metrics there is no way to manage or measure anything. It enables organizations to move past "strong security" claims and pushes towards decisions that are backed and guided by data.

Types and Categories

Security Indicators have two main categories that they are divided into. Deployment or process indicators show whether controls are set into place like devices that run current patches or monitor whether employees/staff have completed cybersecurity trainings. Then there are outcome or effectiveness indicators measure the outcomes and performance such as the time (average) it took to detect and resolve incidents. Some frameworks also include and use risk-exposure indicators to spot potential weak points and vulnerabilities within the systems.

Frameworks and Standardization

The European Telecommunications Standards Institute (ETSI) developed some ISI work items that guide organizations in the selection of indicators, classifying events, and building the technological architectures. In the United States, the National Institute of Standards and Technology (NIST) published, "Special Publication 800-55 Rev. 1", giving guidance on how to build and maintain security measurement systems for the best optimization and protection. These indicators are an integral and crucial part of cybersecurity governance, with these frameworks showing that.

Best Practices for Using Indicators

Experts recommend many different good practices for the efficiency and usefulness of these indicators. Indicators should lead to decisions instead of just describing problems, making them "actionable". Additionally, the indicators shouldn't just focus on technical issues. They should be able to align and understand the business goals. With an abundance of indicators or too much complexity, things become complicated and focus is lost, making simplicity key. Lastly, the indicators should help and allow organizations to compare different performances over time. This benchmarking system leads to better improvement and will allow comparisons to the peers as well. When all these indicators are paired and used correctly, trends could be easily tracked, decisions are justified, and security statuses are communicated clearly.

Challenges and Considerations

Using ISIs effectively to its best usage can be difficult, even with the absolute best practices. A major challenge that is often faced is ensuring that indicators are measuring the correct things. At times, organizations are measuring and tracking the metrics that are easiest to count, but not what is truly affecting them and putting risk to them. Data quality and consistency also raise problems because different organizations or people can view and interpret incidents differently. To have the best results and avoid these problems, experts advise to combine quantitative data with expert judgement that will consistently review and track metrics.

Information security indicators have been standardized by the ETSI Industrial Specification Group (ISG) ISI. These indicators provide the basis to switch from a qualitative to a quantitative culture in IT Security Scope of measurements: External and internal threats (attempt and success), user's deviant behaviours, nonconformities and/or vulnerabilities (software, configuration, behavioural, general security framework). In 2019 the ISG ISI terminated and related standards will be maintained via the ETSI TC CYBER.

The list of Information Security Indicators belongs to the ISI framework that consists of the following eight closely linked Work Items:

1. ISI Indicators (ISI-001-1^[1] and Guide ISI-001-2^[2]): A powerful way to assess security controls level of enforcement and effectiveness (+ benchmarking)
2. ISI Event Model (ISI-002^[3]): A comprehensive security event classification model (taxonomy + representation)
3. ISI Maturity (ISI-003^[4]): Necessary to assess the maturity level regarding overall SIEM capabilities (technology/people/process) and to weigh event detection results. Methodology complemented by ISI-005 (which is a more detailed and case-by-case approach)
4. ISI Guidelines for event detection implementation (ISI-004^[5]): Demonstrate through examples how to produce indicators and how to detect the related events with various means and methods (with classification of use cases/symptoms)
5. ISI Event Stimulation (ISI-005^[6]): Propose a way to produce security events and to test the effectiveness of existing detection means (for major types of events)
6. An ISI-compliant Measurement and Event Management Architecture for Cyber Security and Safety (ISI-006^[7]): This work item focuses on designing a cybersecurity language to model threat intelligence information and enable detection tools interoperability.
7. ISI Guidelines for building and operating a secured SOC (ISI-007^[8]): A set of requirements to build and operate a secured SOC (Security Operations Center) addressing technical, human and process aspects.
8. ISI Description of a whole organization-wide SIEM approach (ISI-008^[9]): A whole SIEM (CERT/SOC based) approach positioning all ISI aspects and specifications.

Preliminary work on information security indicators have been done by the French Club R2GS. The first public set of the ISI standards (security indicators list and event model) have been released in April 2013.

Theoretical Foundation: Security Metrics vs Security Indicators

The term “security metrics” is often used by researchers to show all the ways we measure how secure a system really is. Security indicators are part of the larger group. Metrics include measurements of vulnerabilities, the strength of company defenses, threats that are happening, and outcomes of the incidents. Bigger ideas are broken down and become simpler with security indicators, making things easy for organizations to understand with the help of numbers. For example, indicators might show that “92% of devices were patched within the last two weeks,” instead of saying a system is “fairly projected.” When general ideas become numbers and metrics, organizations are able to be honest with their issues and their security, spotting things a lot quicker. Some organizations use a method called Goal-Question-Metric (GQM), which is when indicators are connected to meaningful goals. This helps companies focus on what truly matters instead of tracking metrics that will never be used or questioned. Now, different environments and systems won’t use the same indicators, it all depends on the system. A traditional office network is not the same as a simple singular smart device, indicators must be tailored to the system and its needs. A generic indicator will not work with every environment, there are differences that should be taken into account and valued.

How Organizations Use Indicators in Practice

Within companies and organizations, there is a lot more that goes into writing a list of things to measure for indicators. A collection and set of organized data is needed to ensure that indicators are accurate, and this is completed with the help of tools. A major tool that is used today is a SIEM system, Security Information and Event Management. SIEM collects logs, alerts, and security events from many different networks and places them within a central location for the organization. This allows for security teams to analyze the findings, gather information, and detect threats in order to be able to put it together and build stronger indicators. Additionally, these SIEM systems track information like the time of an attack or the number of violated policies, that are used within reliable data.

In Europe, there was a full framework explaining how to classify security alerts, build indicators, test the detection systems, and how to evaluate an organization's defenses. This was all created by the ETSI ISI working group which has been helping companies make their indicators consistent and puts them in competition with others. The NIST has a similar guidance which helps maintain and build measurement programs within the United States. This is important because cybersecurity is a crucial issue that only continues to grow in importance with more daily threats. Indicators are heavily depended on to make decisions, ultimately helping all members of an organization see the necessary information in a clear and simple way.

Common Challenges With Security Indicators

Organizations continue to face several challenges when using indicators, despite how helpful they can be. One of the biggest issues is the lack/missing data or the quality of the data. When complete logs aren’t collected by a company or organization, the data will be misleading and will not convey the right message to anyone. For example, if only a portion of the system’s events were collected, the right reflection of “incident counts” or the “detection rate” will not be accurate. Also, when companies choose the wrong thing to measure, good, tangible information cannot be analyzed. It is very easy for some organizations to get caught up in numbers that may be shocking but in reality aren’t genuinely contributing anything about the risks that are being faced.

Another struggle that organizations face is having way too many indicators. When a dashboard becomes overcrowded and there are numbers all over the place, what truly matters becomes difficult to understand. Many recommend choosing few indicators that will clearly convey the risk, performance, and improvement of the system. Within companies and organizations, different risks are faced, therefore indicators must fit with their environment. Places like a bank, hospital, government office, or manufacturing facility will all face different risks and cannot use the same indicators. Companies are urged to design the indicators to match their needs and organization, creating functional indicators that produce real, quality results.

Indicators as Part of Business Strategy

Cybersecurity has become a very important thing that companies and organizations are beginning to take a lot more seriously. It is no longer a technical subject, it has been playing a huge role in how companies are operating, planning, and managing risk. Indicators have transitioned from helping making security decisions to also making business decisions. They have helped companies track progress and improvement of the company over time. Additionally, executives are now using indicators to see how much should be invested into cybersecurity tools, staff, and training. If there is a drop in numbers, like an incident response time, the company knows their investment is paying off the way it should be. When indicators point out slow patching systems and an increase in phishing attempts, executives can make tangible changes to policies, tools, and training, instead of guessing. Security teams now have the opportunity to show how valuable they are to individuals outside of the technical realm, helping them secure more pay and support.

Standardization and the Evolution of ISIs

The ETSI ISI framework is used to standardize security indicators. There are guidelines for events, building indicators, testing detection methods, maturity, and organizing security operations. This allows for organizations to shift from taking qualitative approaches to quantitative ones, with numbers showing what is truly going on. But these guidelines must keep up because cybersecurity is always changing. When technology evolves, things like systems, cloud platforms, devices, and networks need to adapt and be updated to withstand new, modern day risks. Researchers continue to work towards new approaches to make these indicators  flexible, not only for traditional corporate settings but for all types of networks.

Future Trends and Emerging Directions

As technology continues to make improvements, it is important that experts continue to look ahead. Many believe that indicators will start to become predictive, helping see problems before they arise. This will only continue to improve with Artificial Intelligence and Machine Learning, which will help propel these indicators. They may begin to give warnings about vulnerabilities based on past behaviors or patterns that are beginning to occur with suspicious activity. Prioritizing risks and acting early is critical, cybersecurity will only continue to get more complex and organizations will strive to stay up to date and protect their systems.

Conclusion

Overall, information security indicators play a pivotal role in helping organizations understand, improve, and change their cybersecurity structures. When things go from complex to clear, indicators make it easy to track risks, spot weakness, and guide decision making at all levels. Technology will continue to improve and evolve, so organizations and indicators will need to adapt. With the combination of reliable data and specific measurements, information security indicators will continue to be essential for company's protection now and in the future.