Whilst the intentions and objectives behind ISO / IEC 27001:2005 (ISO 27001) aren't dramatically different to those in BS 7799-2:2002, one of the changes with the biggest potential impact to organisations is the requirement to measure the effectiveness of selected controls - or groups of controls - within the new Standard (for more details see ISO 27001 Clause 4.2.2 d).
This new requirement not only demands that businesses specify how these measurements are to be used to assess 'control' effectiveness (there are now 133 controls in the new Standard), but also how to measure the selected controls effectiveness.
In addition to this, the new standard requires that these measurements are comparable and reproducible e.g. so they can be used time and time again, and compared on a year by year basis to gain a better understanding on trends, etc.
So why do organisations need to measure security effectiveness in the first place?
Well, it would be easy to say we're secure, but how can you demonstrate that an organisations security controls are working effectively, and equally importantly - how can you demonstrate:
- Ongoing improvement;
- That your organisation has met legal, regulatory and compliance with standards, contractual requirements such as Sarbanes Oxley - SOX, BASEL II, Payment Card Industry Data Security Standard - PCI DSS, etc;
- That any future expenditure is based on sound and reasoned security solutions, software, hardware, training, etc, that is appropriate and effective within your organisation;
- Your organisation is compliant with ISO 27001 (including other Management Systems such as ISO 9001, Infrastructure Technology Information Library - ITIL®, COSO, CoBIT, ISO 20000);
- Where implemented controls are not effective in meeting their primary objectives - to reduce risk;
- Assurance to audit, senior management and stakeholders that risk justified implemented controls are working effectively (i.e. they have invested their money wisely or a good return on investment - ROI).
You could be forgiven for thinking this should be a reasonably straightforward task. After all, most IT departments throughout the world have been working within some kind of measurement infrastructure (e.g. sometimes measured within contractual key performance indicators - KPI, service level agreements - SLA, operational level agreements - OLA) since the mid-1990’s and should, by rights, used to considering how to measure their IT effectiveness (most IT departments I've known usually have very stringent methods for providing value for money).
The challenge is, the whole area of how, and what is good and effective security is subjective and therefore difficult to quantify, let alone provide statistics.
In actual fact, most organisations there is often plenty of evidence that good security practices and controls are in place, especially those organisations that have already implemented management frameworks such as CoBIT, ITIL® or COSO.
So what are the benefits of measuring your organisations security effectiveness?
- Provides real tangible evidence of cost reduction - through better risk management and reduction of impact caused by exploitation of threats;
- Provides better cost / benefit analysis and therefore helps ensure ROI decisions going forward;
- Actually eases process of monitoring the effectiveness of the ISMS (e.g. less labour intensive, for example, if using tools, and provides a means of self checking);
- Proactive tools to measure can prevent problems arising at a later date (e.g. network bottlenecks, disk clutter, development of poor human practices);
- Reduction of incidents and better understanding of root cause;
- Motivates staff when senior management set targets;
- Tangible evidence to auditors, and assurance to senior management that you are in control - i.e. corporate information assurance (corporate governance), and top down approach to information assurance.
Whatever the organisational drivers for measuring the effectiveness of security, it should no longer be just about identifying the controls to be implemented (based on the risk assessment), but also about how each control will be measured against its original objective (to reduce the chances of the risk being exposed). After all, if you can’t measure it, how do you know it’s working effectively?
Before deciding which control should be used to measure effectiveness, your organisation should undertake ensure the following activities first:
- Confirm relevance of selected controls through risk assessment (Mandatory requirement anyway for both ISO 27001 compliance and certification projects) ;
- Define objectives, ensuring they map back to the business objectives;
- Use existing Indicators wherever possible, e.g. in ITIL® terms, KPIs:
- A KPI helps a business define and measure progress towards a particular goal;
- KPIs are quantifiable measurements of the improvement in performing the activity that is critical to the success of the business.
- Within the information security management system (ISMS) audit framework, identify controls which can be continuously monitored, using a chosen technique;
- Establish a baseline, (e.g. we security awareness trained x amount of people in y timeframe) against which all future measurements can be contrasted/compared;
- Provide periodic reports to appropriate management forum / ISMS owners (show graphs, pictures paint a thousand words);
- Identify Review Input - agreed recommendations, corrective actions, etc;
- Implement improvements in line with any existing management systems e.g. ISO’s 9001, 14000, 27001, 20000, 18000.
- Establish / agree new baseline, review the output, apply the PDCA approach (i.e. Plan - Do - Check - Act).
Hopefully, each business may have its own measurements already in place (e.g. SLAs, OLAs, KPIs), the challenge is to set a 'measurement', which is realistic based on a previous known figure, and ensure the future figure is measurable and reproducible.
Senior management and possibly auditors, are more likely to want to see the big picture, therefore, consider monitoring the effectiveness of a group of controls, e.g. Section 13 of ISO27001 - Security Incident Management, plus others.
Try to encourage senior managers to buy into realistic first-time goals, such as measuring how well you coped with the latest security incident and in what time frames did you hold the lesson's learnt meeting and at what point did the improvements to the existing system become operational.
When selecting controls to be measured for their effectiveness, it is worth bearing in mind that you could group them into four categories:
- Management controls
Security policy, IT policies, security procedures, business continuity plans, security improvement plans, business objectives, management reviews; - Business processes
Risk assessment and risk treatment management process, human resource process, SOA selection process, media handling process; - Operational controls
Operational procedures, change control, problem management, capacity management, release management, back-up, secure disposal, equipment off-site; - Technical controls
Patch management, antivirus controls, IDS, firewall, content filtering.
In a changing environment, new baselines will need to be set each time a major change or incident occurs within the ISMS, so this is just the beginning. Try to establish regular review cycles of your security effectiveness measurements and consider how this might improve, how your organisation can become more effective in the management of its incidents.
Above all, it should start to dispel rumours that security and risk management is a black art and that it is unmeasurable. In fact, we should start to see tangible benefits from measuring and improving our ISMS's and security management solutions globally.
For more information, the new ISO27004 will soon be published, this will help those organisations who are unclear about this and this new standard should help any organisation get a grip on measure the effectiveness of security.
Good luck.
