Jennifer Vogl (jennifer.vogl@cdw.com) is a Governance and Compliance Manager at CDW in Madison, Wisconsin, USA.
As someone on a compliance team will tell you, the question “Will we pass the audit?” is not uncommon. That question gets asked in organizations with robust, ongoing compliance programs, as well as in organizations with less formal programs; though, it probably happens more often in the latter. It could be a technical subject matter expert (SME) asking that question, or it may be a member of executive leadership. In any of those circumstances, the compliance team (or person wearing that hat) will probably have an answer that fits somewhere between, “We’ll almost definitely pass,” and “Well…we’ll only pass if we fix the following issues.” But where does the compliance team get its answers? Whether formally measured or casually monitored, those answers come from key compliance indicators (KCIs). These KCIs can be identified to benefit organizations of any size with compliance programs of varying levels of maturity. Whether your organization has an informal compliance program or a robust one, KCIs can be monitored to drive changes that reduce the risk of audit findings, incrementally mature the overall compliance program, and provide some peace of mind.
What are key compliance indicators?
KCIs, as the name suggests, are the operational and organizational clues that provide assurance that the activities required for meeting specific requirements are happening the way they should be. Those clues can be related to operational tasks, resourcing, the culture of an organization, or any of the activities necessary to achieve compliance. Specifically, a key compliance indicator is a metric or measurement that provides a quantitative description of an organization’s adherence to a stated compliance objective.
While not necessary for understanding KCIs, those familiar with key performance indicators (KPIs) may see some parallels. Both KPIs and KCIs are ways to measure success against objectives. For KPIs, those objectives are related to business goals. For KCIs, those objectives are related to compliance. As we examine this concept, it will become clear that the compliance objectives must be very specific. For example, having a compliance objective to “pass the audit” is not going to yield much value without further exploration and definition. Success with KCIs lies in specificity. The answer to the “Will we pass the audit?” question should be an aggregate of the answers to more specific compliance objectives like, “Can we demonstrate a sufficient access management program?” and “Do we have adequate separation of duties?”
KCIs may vary from company to company, even where the same compliance objectives exist. As we drill down through the levels of specificity to get to our KCIs, the specific tasks that comprise each implemented control will guide the identification of the KCIs. While some KCIs can be tied to automations related to continuous auditing, it’s also just as likely that the indicator could be derived from an implied requirement or even from a control implemented as a remediation after gaps were identified.
How do we identify compliance objectives?
As we’ve already discussed, our compliance objectives determine our KCIs. Depending on the audit type, that compliance objective may come directly from the standards, themselves. For example, a payment card industry audit is, basically, an open-book test, as the standards are accompanied by guidance material, objectives, measures, etc. developed by the same people that develop the standards. That guidance material provides the context for each requirement, the overall goal, and a description of how it should be audited. For other frameworks, the compliance objectives may be a little less prescriptive, but they can still be identified by examining the explicit and implied control requirements, along with industry guidance related to that control. Compliance objectives can also be learned through previous audit experience and the testing procedures experienced firsthand. Essentially, though, using whatever guidance and experience is available, we all need to answer the basic question, “How will we know when we’ve succeeded or failed at meeting a compliance objective?”
Ask ‘how’ questions
For any stated compliance objective, measurable indicators can be derived by asking the question, “How will we know when we’ve done it?” In other words, what are our symptoms of success? Let’s walk through an example that applies to multiple industry frameworks and regulatory obligations: access management.
Compliance objective: Maintain an access management program that ensures that only people who have a business need have access and where access is removed timely after that business need has expired.
In this state, the objective is very broad and not very measurable. Let’s use a concept similar to the commonly understood “5 Whys,” or root cause analysis, but instead of using “why,” we are going to keep asking ourselves, “How?” We get two main paths from the stated compliance objective, and both of those could easily yield KCIs. Those paths are as follows:
-
“How do we ensure access is only obtained by people who have business needs?” The first set of “how” answers might include:
-
Access must be formally requested.
-
Access requests must be approved.
-
Onboarded employees only get certain access based on role, etc.
-
-
“How do we ensure access is removed timely after that business need has expired?” The first set of “how” answers might include:
-
Access is removed within 24 hours of termination.
-
Access is removed within five business days of job change.
-
Accounts and rights are reviewed regularly, etc.
-
Are we at a point where we have identified measurable KCIs yet? Not quite. One or two more rounds of “how” might help us get there. But let’s follow just one of these paths for the sake of simplicity.
“How will we know that access is removed within 24 hours of termination?” The following are possible answers:
-
Direct managers submit access termination requests to human resources within one hour of the time and date of formal termination/exit.
-
Accounts in the access/identity management tool are made inactive within 23 hours of the submission of the termination ticket/request.
-
Physical badges are made inactive within 23 hours of the submission of the termination ticket/request.
Then, “How will we know that accounts in the access/identity management tool are made inactive within 23 hours of the submission of the termination ticket/request?” Answer:
-
100% (or choose your tolerance) of the accounts deactivated in the access/identity management tool have a time stamp of within 23 hours of the submission time stamp on the termination request.
This last measurement can be automated, using data from existing access management tools, or manually, using some sort of verification for each termination for a sample of terminations over a given period. Whether automated or manual, this measurement is one of your KCIs for access management. The percentage of success each time you measure it is your window into the likelihood that this part of the access management requirement is being met regularly.
What should we monitor?
For organizations that must adhere to regulatory obligations, industry standards, or even operational requirements to maintain partnerships, the number of KCIs, just like the number of specific requirements, can be in the hundreds or even thousands. And from the access management example, it’s clear there are probably half a dozen KCIs we could derive from that one compliance objective. Is it necessary, or even desirable, to monitor every possible compliance indicator? That answer is a clear and resounding, “No.” Just like any use of metrics, reporting, or dashboards, too much data can distract from the important issues and will facilitate the feeling that the organization is collecting metrics for metrics’ sake. Instead, KCIs should be focused on areas that are likely to cause compliance failures, and they should be tailored to the specific success criteria for the requirement.
Good candidates for KCIs include those compliance objectives for which:
-
Automation and exception reporting are not possible or feasible such that adherence relies on manually executed tasks. These are the types of tasks that can fail more frequently and without automated visibility and response to a control failure. KCIs provide that visibility and opportunity for quick issue resolution.
-
Gaps or issues have been previously identified by auditors, so scrutiny is increased both internally and by future auditors. In these cases, the KCI might be derived successfully from the objective of whatever remediation was implemented, as well as from the original compliance objective.
-
Fault tolerance is low. For requirements that include tight and inflexible deadlines and where the failure to meet those deadlines equates to a compliance finding, KCIs can be used to insert a “buffer zone” where exceeding an internal threshold will generate an alert and action to prevent breaching the actual compliance deadline.
KCIs can be used in any of these areas to provide early warning systems, evidence that can be used across multiple frameworks with similar compliance objectives, or even just the peace of mind that comes from knowing that compliance objectives are being met. Remember, as your compliance program changes, the requirements or solutions change, or your areas of concern change, so will your KCIs. Even if your organization only implements one KCI, it’s useful to revisit that metric regularly to make sure that’s still the important metric and it’s still using the right data to predict audit success.
Moving forward
When building or maturing a compliance program of any size, it’s easy to feel like most improvement steps require sweeping operational changes, complex solutions, or some sort of automation. However, KCIs can be focused, granular, and implemented one at a time, making them a great way to increase the maturity of a program or build in some compliance assurances in small, manageable bites. Organizations of any size with any level of compliance maturity can benefit from identifying one or more KCIs. Whether your organization has a small, lean compliance program and KCIs are identified in only a few critical areas or your organization has a large, robust compliance program with multiple, well-placed KCIs, the metrics provided are used to either provide assurance or drive action/issue resolution. So, will you pass the audit? If that answer causes any hesitation, the measurement and reporting of one or more KCIs could help your organization breathe a little easier.
Takeaways
-
Key compliance indicators (KCIs) are to regulatory obligations what key performance indicators are to the bottom line.
-
KCIs can provide advance warning of possible compliance gaps.
-
KCIs advance the maturity of the programs and controls implemented for both security and compliance.
-
KCIs are beneficial for all types of organizations with any level of compliance program maturity.
-
KCIs aren’t just metrics for metrics’ sake. Get to the right measurement with asking the “hows.”