Duncan Milne (dmilne@bupalatinamerica.com) is Chief Risk and Compliance Officer for Bupa Global Latin America in Miami, Florida.
With the breadth of most compliance programs in large international organizations and the number of policies and reports to follow, it can be easy to lose focus of the bigger picture and fall into the trap of measuring the input—how many reports or spreadsheets did I issue; how many training sessions did I run—rather than the output and whether your program is meeting its objectives and actually having any positive impact on the business.
Regulatory expectations in this area are increasing, and when setting compliance metrics, we should always challenge ourselves about not only what any particular metric is trying to prove, but also why that metric is relevant and aligned with the wider business goals.
When going through this exercise, it can be useful to look at metrics through the lens of four different levels, and ideally, your program will contain a balance of each one to be able to prove that the program is functioning and understood, affecting behaviors, meeting its goals, and adding value to the organization.
Operational metrics
The most basic level of metric are operational metrics. These are usually quantitative and record a factual event (for example, the number of compliance training sessions that have been held).
We typically use operational metrics to prove that the program has been implemented and is functioning—for example, to track the dedicated number of resources in place or the number of whistleblower reports that have been received. These metrics can be a useful foundation, but on their own, they don’t really tell you very much and are only a starting point to then link to the more qualitative aspects of whether a compliance program is actually effective.
Engagement metrics
Engagement metrics are essentially asking three questions:
-
Are people aware of the program?
-
Do they understand the program?
-
Do they care about it?
In this space, you can find both qualitative and quantitative metrics. The use of surveys and questionnaires can help to identify levels of understanding and awareness and track them over time, helping to identify particular departments or seniority levels that might require more focus.
Quantitative data can also be used to measure engagement, and this is an area where marketing teams have particular expertise (and may be able to help) through measuring customer engagement by way of social media hits and recording site traffic.
From a compliance perspective, engagement metrics will typically track the level of staff interaction with the content that your program is generating: the percentage of compliance communications that are opened, the page views for your intranet, time spent on the page, and repeat visitors to the page.
Again, this information is useful but not in isolation, and while staff might be aware of, understand, and care about the program, they still may not be doing anything differently as a result.
Behavioral metrics
Behavioral metrics link back to your original goals for the compliance program and measure whether corporate behaviors and culture are having the desired effect.
Many aspects of a compliance program are linked to influencing or changing behavior. We might be looking to increase the number of people reporting compliance incidents, improve training scores, or encourage people to report conflicts of interest or gifts and hospitality.
When setting these metrics, we should first articulate what it is that people are expected to do differently from before, and then the relevant metrics will show whether this is actually happening.
Business value metrics
We may have a compliance program that we can prove is operating, understood by staff, and affecting behaviors, but it doesn’t operate in a vacuum, and perhaps the most important metric level of all is the final one: those metrics that demonstrate that the compliance program is meeting its objectives and adding value to the business in which it operates.
The compliance program is not an end in and of itself and should not become introspective. It is part of a wider business, consumer, and regulatory environment and should be a valuable part of the business strategy. The more we as compliance officers can show this and align ourselves with organizational metrics, the greater prominence and influence the program may generate.
Business value metrics can take many forms. For example, some metrics might highlight the trend of health and safety incidents—in particular, demonstrating how, as a result of the training or through root cause analysis from incident reporting, the number of health and safety issues went down by a certain percentage. Or similarly, metrics showing a decrease of code or regulatory breaches.
This level of thinking can be also applied to a number of business performance metrics, such as showing the impact of the compliance program on business process improvements, operational efficiencies, customer satisfaction (for example through net promotor score surveys), and even the bottom line.
Ideally, a balanced selection of metrics can be found across all different levels, and in many respects, it is important to be able to demonstrate all of them to be able to show the necessary cause and effect. But the more we can use metrics to understand how the program is working effectively and how it aligns with the goals of the program and business strategy, the more well-rounded insight we will have, which will allow us to continuously identify areas for focus and enhancement.
Takeaways
-
It’s key to be able to employ metrics that measure not just input and production but also whether the program is meeting objectives and adding value.
-
Be clear on objectives and target behaviors at the outset to track whether these are being achieved.
-
Challenge what each metric is trying to show and why that metric is relevant.
-
Align compliance goals and metrics with wider organizational goals, and use tangible metrics that show these shared goals.
-
Employ a balance of operational, engagement, behavioral, and business value metrics to understand the full impact of your program and identify where to improve.