Why do we need metrics?

Here are a few reasons why metrics are so vital to your program’s success:

• Metrics help to measure whether the program is effective and meeting its objectives.

• They provide transparency to the leadership team by giving them visibility into the work done by ambassadors to drive compliance deliverables.

• Metrics demonstrate the value of this program to internal and external stakeholders (e.g., executives, board of directors, auditors, regulators).

• They show how well the program is accepted by employees and the cooperation it receives from different business areas.

• Metrics enable us to proactively identify control gaps, address risks, implement remedial actions, and measure how the program evolves over time.

Despite the best intentions and efforts, things may sometimes not work as planned, some employees may not wholeheartedly support the program.

Escalating such noncooperation can be a sensitive matter. Having clearly defined metrics provides an objective and fact-basedway to highlight business units or countries where the program is not working well and get support from the management to push for greater participation.

Using metrics to develop dashboards and reports gives useful information that can be analyzed to perform root cause analysis of issues and prioritize where to direct more resources to drive improvements.

Where to start

The following are some things to consider when setting up metrics:

• Consider the rationale behind setting up the program and the goals it seeks to achieve.

• Set up metrics to track and benchmark goals in accordance with the relevant regulatory and industry requirements.

• Use a mix of quantitative and qualitative metrics to measure the program’s impact.

• Consider how metrics will be recorded and what evidence can be maintained.

• Think about how to convey a story by tying the data to the impact created by the program.

Metrics can’t exist in isolation. While defining the metrics, challenge yourself with the question, “So what?” This reminds you to consider what activities the metrics measure, what goals they hope to achieve, and how they make the program more effective.

Define clear goals right at the beginning of the program

It is also helpful to clearly identify and define your goals at the start.

• Clearly identify the objectives the program was set up for in the first place.

• Are there any issues or challenges you are looking to solve (e.g., technical, regulatory, ethical, cultural, or behavioral issues; driving change and improvements to the ethics and compliance program)?

• How would you measure and show evidence of the positive impact being created?

• Link the metrics to specific goals.

• Build alignment with management to have a common vision of what the program seeks to achieve. If you get management buy-in at the start and agree on what success should look like, you are more likely to have their backing, which is helpful when asking for a time commitment from people.

• Establish a benchmark of the goals you want to achieve, and work backwards from those to set milestones along the way.

The end destination may well change with time (or new goals may get added), but it’s essential to retain visibility on the original goal for consistency.

Examples of goals for the ambassadors program

The following are some potential goals for your ambassadors program:

• Expanding the reach of the ethics and compliance program over a wide base of employees.

• Strengthening the ethics and compliance culture.

• Identifying cumbersome or inefficient compliance processes and driving process improvements.

• Avoiding regulatory issues.

• Imparting meaningful risk-based and role-based training for staff with higher compliance risk roles.

• Enhancing the speak-up culture to encourage employees to report issues safely and assure them of nonretaliation.

• Improving health and safety practices by timely reporting of incidents.

• Seeking frontline feedback on how well the compliance messaging is being accepted within the workforce and how ethics and compliance is getting embedded throughout the day-to-day operations to become part of the organization’s DNA.

Figure 1: Four categories of metrics

Metrics can be of various types, and we can sort them into four categories (Figure 1).

Operating metrics are tangible and quantifiable metrics.

• Number of compliance ambassadors appointed and business areas they represent.

• Geographic reach of the ambassadors.

• Number of training sessions conducted and percentage of attendance achieved.

• Number of queries handled or number of projects supported by ambassadors.

• Average tenure of ambassadors in this capacity.

Engagement metrics are a mixture of quantitative and qualitative metrics that can be tracked to evaluate whether people are aware of the program, if they understand it, if they are engaging with it, and if they care about it.

Some quantitative engagement metrics are:

• Percentage of compliance communications that were opened and read.

• Page views for the compliance intranet page.

• Average time spent on page.

• Number of repeat visitors to the page.

Some qualitative engagement metrics are:

• Sample staff survey on awareness and understanding of the program.

• Comments on blog posts and message boards.

• Post-training quiz results.

• Results from culture surveys that capture the perception of staff at different levels.

Behavioral change metrics link to the “why” of the program (i.e., in what ways have ambassadors helped enhance the culture of ethics and compliance).

A few examples of behavioral metrics are:

• Number of reports made to the ethics hotline.

• Process noncompliance or integrity violations raised by or through ambassadors.

• Rate of disclosures made for potential conflict of interest cases.

• Timely closure of remediation actions.

• What is the rate and trend of speak-up reporting?

• Number of reports of retaliation against those who speak up.

• Time lapsed between events occurring and the compliance team becoming aware.

This kind of data allows us to do a trend analysis to see if there are particular business units or countries that have a higher incidence of reports, the types of issues reported, how many were substantiated, what was the root cause, and what were the outcomes and post-remediation actions put in place. The data also helps to determine additional resource requirements and makes a business case by highlighting problem areas as well as showcasing improvements already being made by engaging ambassadors.

Business value metrics are the final level of metrics where we can correlate the work done by ambassadors with the impact on the business.

• Decreasing rate of health and safety incidents (i.e., showing that due to the increase in training or incident reporting as an outcome of the ambassadors program, health and safety breaches went down by X%).

• Showing a decreasing trend in the number of violations of the code of conduct, compliance policies, or regulations by comparing pre-program numbers with those after program implementation.

• Showing business process improvements and operational efficiencies that resulted from issues highlighted by or via the ambassadors.

• Seeing a positive trend in customer satisfaction surveys (e.g., through “net promoter score” surveys).

• Increasing efficiency and improvements to the bottom line.

With all these different metrics, we should be thinking about finding the right combination to demonstrate that the program is successful.

It is also necessary to keep in mind that there will, of course, be some blurring between the cause and effect. For instance, there could be other factors as to why the number of compliance process violations decreased besides the increased awareness created through compliance training sessions. Nonetheless, it is worth persevering and tracking these metrics, as it can lead to wider acceptance and greater support.

Maintaining evidence to support metrics

For a compliance program to be effective, a key element is documentation and good record-keeping. Right at the beginning, when the metrics are being defined, we must remember the types of documentary evidence we can have to support the metrics.

• Log of noncompliance with processes and integrity violations.

• Tracker for queries handled by ambassadors.

• Reports of training sessions imparted and communications sent out by ambassadors.

• Tracker for remediation actions.

• Culture scores: formal and informal surveys, conducted by ambassadors and compliance professionals, of different employee groups.

• Log of successful implementation of compliance pilots and projects where ambassadors were involved.

• Log of improvements by comparing data from pre- and post-program rollout (e.g., sharing examples of processes that were simplified after receiving feedback from or through ambassadors).

Report progress to management

While determining the metrics, we should simultaneously be thinking about the following:

• How all data points will get retrieved and collated, such as data on compliance training sessions, remediation actions, investigations, and noncompliant processes.

• How will reports be generated? Will data be collected manually through different sources, or will it be possible to easily collate relevant data through a tool and generate dashboards and compliance scorecards?

• In which forums will these reports be shared with management? Is there a governance committee meeting or any other forum, like business review meetings, that can be leveraged?

• How frequently should metrics be reported (ad hoc, monthly, quarterly, or biannually)?

To provide assurance to management that the program is working well, we must give regular updates on the progress being made, such as:

• Achievement of goals: What program milestones have been achieved in line with the agreed goals, and what value is that delivering to the business?

• Positive messaging: In addition to compliance professionals, ambassadors could also share stories about issues they helped to resolve, changes they were able to influence, or challenges they faced and overcame. This instills greater trust in the program’s relevance and effectiveness.

• Metrics and improvement trends reports: Sharing these reinforces the message that ambassadors are a cost-effective way to extend the compliance footprint across the company.

• Examples of conversations: Talk about how colleagues discussed issues with ambassadors, which they would otherwise have been reluctant to escalate, to demonstrate the value of the program and the level of its acceptance by the staff.

• Significant contributions: Explain how ambassadors contributed to implementing and embedding the compliance program in cases where a new entity was acquired, or new offices were set up.

• Examples of compliance promotion: Ambassadors can serve as compliance evangelists to enhance the image of the compliance function as a business enabler. They can share examples of modifications that were made to the ethics and compliance program. This also assures the management team that the program is practical, useful, and evolving.

Keep the program relevant and dynamic

As with any other new initiative, there can be a risk that the program starts strongly and then fades over time. It may become stale, and people might get bored, complacent, or simply overtaken by the next new project. The program must be refreshed regularly to ensure it has the right impact. For this, the following is important:

• Actively listen to feedback from ambassadors to understand how they are experiencing their new roles, particularly on the front line.

• Identify success stories from ambassadors and share these through a strong communications campaign to generate enthusiasm, maintain momentum, and bring the program to life. For example, you might run a monthly communication where you spotlight a particular ambassador’s contributions and do a question-and-answer session with them to highlight their specific achievements.

• Seek feedback from other stakeholders and try to link it to the customer as much as possible because what is good for the customer is usually good for the business.

• Identify and remove blockers. What is stopping people from reporting incidents? Is it fear of retribution, a general reluctance that reporting might create extra work, or concerns that there will be no change as an outcome of reporting?

• Refine and tailor the program to keep it relevant as the compliance program evolves.

• Use strong communication and storytelling to keep the buzz alive and attract new ambassadors.

• Use rewards and recognition programs to help motivate ambassadors (e.g., highlight an ambassador who has been most active in driving improvements).

If you can formalize the ambassadors’ responsibilities by making them part of annual performance objectives, then the ambassadors’ level of engagement—and the resultant positive changes they bring about—become more likely to be sustainable.

An effective compliance ambassadors program can provide valuable insights across the whole organization and help you understand the challenges that need to be overcome to achieve the objectives of the overall ethics and compliance program.

Takeaways

• Establish clear and specific goals for the program and gain alignment from the leadership team.

• While determining metrics, keep in mind how they can be linked with the agreed-upon goals.

• Having a good mix of quantitative and qualitative metrics effectively demonstrates the goals of the program that have been achieved and their impact.

• Metrics should clearly show the correlation between improved business performance and how this has been influenced by achieving the program’s goals.

• Data from tracking metrics and incorporating feedback from the ambassadors and other stakeholders can keep the program dynamic and relevant.

This article is based on a presentation co-developed with Duncan Milne, associate general counsel for Global Benefits Group. We jointly presented this during SCCE conferences in 2021 and 2022.^[1]

Opinions expressed here are solely those of the author and do not express the views or opinions of her past employers.