An Easier Way to Understand the Effectiveness of Risk Controls

KRIs, KPIs, ORSA, ISO, COSO…risk controls, risk owners, risk appetite.

The acronyms, the alphabet soup, oh my!

To anyone with little to no experience, risk management jargon can be dizzying and confusing, especially to executives who are often deluged with risk registers, reports, and processes that are overwhelming and not helpful for managing the organization for success. While unfortunate, it should not come as a surprise that fewer than 1 in 5 organizations believe their risk management processes provide a unique competitive or strategic advantage.

After all, executives are not interested in jargon and processes. What they’re most interested in knowing is whether they are on track to meet goals.

Key risk indicators (KRIs) focused on objectives help managers and executives understand this, but when used to their full potential, they can also help the company see what’s coming down the road and take steps to address future uncertainty. Taking proactive steps like this is one of several key differences between traditional and enterprise risk management.

A recent webinar from RIMS and Resolver explored another indicator organizations can use known as key control indicators, or KCIs. In layman’s terms, a KCI is a metric to understand the effectiveness of risk controls, which are actions taken to mitigate the negative impacts of risks.

Other metrics like key performance indicators (KPIs) and KRIs can be lagging and leading indicators. KCIs fall in the middle. In the webinar, Chase Clelland, VP of ERM at Grow Financial Credit Union, provides the following example of what indicators look like for the identified risk of loan defaults:

  • KPI = number of loans for clients who have past defaults
  • KCI = number of clients with insufficient collateral cover
  • KRI = number of loans who have past defaults and do not have sufficient collateral cover

Upper and lower thresholds can be established around each of these indicators that decision-makers can use to change goals, or in the case of Chase’s example, lending decisions or collateral requirements. A post from earlier this year on developing KRIs includes a couple of graphs showing how thresholds around revenue can influence decisions.

Like KRIs, KCIs shouldn’t be developed until after a risk’s likelihood, impact, and any other dimensions have been fully assessed and understood. And like KRIs…

Developing true KCIs to monitor the effectiveness of risk controls can be challenging and should only be done by organizations with robust risk management capabilities.

When asked about the biggest barriers to having effective metrics, Chase identifies the three biggest challenges he has faced: identifying the right metrics to monitor, knowing which metrics to evaluate over time, and having insufficient internal processes.

But in order to have a true KCI, you must consider both inherent and residual risk in the following way – Inherent risk – Control effectiveness (KCI) = Residual risk.

Merriam-Webster defines inherent as:

…involved in the constitution or essential character of something: belonging by nature or habit.

When it comes to risk, it can be complicated to understand what “inherent” means since it requires the manager or executive to imagine an alternate reality where no mitigations or other risk management activities take place.

Therefore, it can be extremely complicated and challenging to understand the organization’s inherent risks, making it difficult to have an accurate residual risk assessment.

So how can I understand the effectiveness of risk controls without complicating things?

Like ERM in general, KRIs and KCIs may sound simple in theory but are far from easy.

When asked about measuring the effectiveness of a control, co-presenter Terry Lampropoulous, Professor of Risk Management at Seneca College, says that it varies by company, industry, and even country since regulations can differ.

Besides testing by the audit group or some other independent party, paying close attention to losses or other indicators can indicate a control is not working. Chase explains that if you see losses going up, you either have “…controls that are not properly in place or a risk assessment that is way off.”

To add to Terry and Chase’s comments, true KCIs may not be necessary or even practical considering the challenges of understanding inherent risks.

The truth is you’re going to have different types of controls, whether they are systems-based, process-based, manual or automated. Besides testing and auditing by a third-party, having a clear understanding of the objectives and carefully monitoring the risk each control is linked to is the simplest way to understand the effectiveness of risk controls.

If it appears like a risk event is materializing or performance metrics are not being met, it may be a sign that the controls need to be changed or the person(s) responsible for implementing them is not fulfilling their obligations.

Although the webinar was quite informative, it was still very risk-centric, or focused on reducing negative consequences and preventing failure. As I explained earlier and in other posts, the point of KRIs and thresholds is not always to prevent a negative situation, but to indicate where additional risk can be taken in a responsible way.

It’s important to proceed with caution when developing metrics, especially a KCI. If your company is not equipped to process, analyze, and take action with the data, the whole process can be more trouble than it’s worth.

A more effective course of action is to understand what your goals are, what can prevent you from achieving those goals, carefully monitoring the steps your company is taking, and adapting your approach as conditions warrant.

How do you measure the effectiveness of risk controls in your organization?

I’m interested in hearing your thoughts on this topic, and I know others in our profession are too. To share your experience, questions, or concerns, please leave a comment below or join the conversation on LinkedIn.

And if your company is struggling to understand how to better assess risks or adapt to ever changing circumstances, please don’t delay – reach out to discuss your situation today.

Sign Up For Our Newsletter

Sign Up For Our Newsletter


Meet Carol

Helping companies achieve their vision and strategy, and succeeding in today's turbulent world, is something I'm honored to be a part of. Whether you're an occasional blog visitor or a long-term client, thank you for letting us be a part of your journey.

Most Recent Posts

The 12 Days of ERM Christmas

Without a doubt, one of my family’s favorite holidays is Christmas. Part of the fun, especially for our son, is seeing what “Santa” brought, but most importantly, we treasure the spirit of peace and goodwill the season brings. And after what seemed to be a never-ending warm spell, the weather is expected to be good…

Read More

Don’t Let Goals and Initiatives Be Blindsided by External Events

As the end of the year draws near, I think we’d all agree that while it wasn’t without its challenges, this year also wasn’t quite as turbulent as the previous two. While a lot of people are juggling company parties, shopping for friends and family, and special activities for the kids, most companies are putting…

Read More

Going the Distance: Ensuring Successful Execution of Strategic and Annual Initiatives

Strategic planning is a challenge – of all people, I understand… After all the meetings, risk and data analysis, and brainstorming of the preceding months, it’s tempting to think this is the end of the road and you can relax. Contrary to this common perception though, this is exactly not the time to relax, but…

Read More

Avoid Rookie Mistakes and Protect your Internal Reputation

Be honest – have you ever done something that you soon realized was a real rookie mistake? Me raising my hand… Considering the nature of ERM’s role to ask questions and challenge assumptions (often during conversations with executives), it can be argued that, in at least some cases, the expectations bar for risk professionals is…

Read More

ERM at Thanksgiving – An Illustration of Risk Management in Action

On occasion, I like to take some of the concepts we risk professionals think about in our jobs and apply them to different personal situations…take some of the same concepts we use when working with executives to develop corporate strategy and manage risks or uncertainty around that strategy. It’s Thanksgiving week in the U.S. –…

Read More

Why Quantitative Risk Assessment is Not Just the Best But the Only Option – A Conversation

Periodically, I have the pleasure of speaking one-on-one with Hans Læssøe on a variety of topics around ERM, strategic risk, and other issues and trends. As you know from my previous conversations (here, here) and posts featuring his work, Hans was formerly a practitioner at the iconic LEGO Company, but even more notably, is a…

Read More

The Three Lines Model – 3 Reasons Why I Don’t Like It

Everyone likes a clear-cut template that offers an easy way to create or manage something…I mean what’s not to like about a step-by-step process for accomplishing what you want? Sometimes this can work without any issues, such as the case with the Project Management Book of Knowledge (PMBOK), ISO 9001 standard, or a new cooking…

Read More

5 Avenues for Expanding your ERM Knowledge

One thing I was taught to appreciate from a young age was the value of education and knowledge. It didn’t necessarily matter what the subject was, just that I always maintain a learning or growth mindset regardless of my current status in life. This mindset has served me well over the years, and it’s a…

Read More

Storytelling and Risk Management – Developing Skills that Technology Cannot Replace

It’s amazing how technology has developed and changed our working world over time. Imagine trying to run my risk and strategy consulting firm without tools like Zoom, Box, Slack, and other ERM-specific technology tools. There is no way we would be able to serve our clients the way that we do. Just consider how the…

Read More

3 Phases to Creating and Launching an ERM Program Focused on Organizational Success

If you’ve been handed the task of creating an ERM program for your organization, let me first offer my congratulations quickly followed by my empathy for the task ahead of you. I don’t say that to scare you but to provide a small dose of reality. Building, launching, and refining an ERM program that is…

Read More