Qualitative vs. Quantitative Risk Assessment – Can There Be a Middle Road?

In my years as both a risk practitioner and ERM consultant, there has and continues to be intense debate around methods for assessing risks. (If you are unfamiliar with the meaning of qualitative and quantitative, this article provides a quick overview.)

To illustrate this debate, take the following two comments from a previous article asking whether quantitative is the only future of risk management:

As you can see from just these two comments on a blog post, it’s interesting the different perspectives you will run into, and like so many issues in today’s world, some of comments I’ve come across can be a little on the mean side. (Why can’t people provide constructive criticism without being mean?)

Qualitative methods can be easier to implement and maintain, especially for companies without strong modeling and statistical analysis capabilities. As I’ve seen over and over, qualitative risk assessment is an option for many organizations starting out simply because they will be overwhelmed if they jump into a full quantitative approach.

This overwhelm can be a huge setback for any risk management initiative that can possibly take years to dig out from…

But as promoters of a quantitative approach contend, qualitative-based risk assessment can be fraught with biases and even be dangerous since you are relying on subjective impulses of the person(s) giving the score as opposed to objective numbers, or as psychologist and Nobel prize winner Daniel Kahneman explains:

Overconfident professionals sincerely believe they have expertise, act as experts and look like experts. You will have to struggle to remind yourself that they may be in the grip of an illusion.

Of course, I’m not going to argue too much about the shortcomings of qualitative assessments, especially when you throw heat maps and risk matrices into the mix, because it is true – there are certainly downsides to using the method, some of which I have personally experienced. Honestly, I haven’t attempted to use a heat map since the early days of my ERM career, because as Douglas Hubbard says in this presentation, tools like this provide no clear answer to the question “Should we spend $X to reduce risk Y or $A to reduce risk B?”

But when I speak with companies, it becomes clear how most of them simply aren’t equipped to take full advantage of quantitative risk assessment methods, modeling, and so on. Fortunately, I have found that…

There’s a middle way to obtain probability ranges without extensive statistics experience.

A strict qualitative assessment method that relies on a 1-5 or low-medium-high scale is certainly fraught with all kinds of pitfalls – I get it.

However, as I’ve explained in other posts, sometimes this is the best you’ll be able to do, especially in the beginning stages of any ERM initiatives. In this case, you do what you can, knowing and communicating that the method is not permanent. As the program matures and capabilities grow, you and executives will need to move into more sophisticated assessment methods.

One approach that I use with some of my clients (…many of whom fall into this situation) is to assign specific criteria behind the scale/rating.

You see, one of the criticisms of qualitative (which I echo) is that the labels of “low”, “medium”, and “high” can mean different things to different people.

However, when you set probability ranges for each rating levels on the scale, or specific attributes behind what a level 3 impact means, you give the executive, manager, or information user a clear idea of what a specific ranking means in the context of a specific risk.

For example, when surveying the likelihood of risk(s) for a client, having probability ranges of <10%, 10-40%, 40-60%, 60-90%, and >90% can provide more clarity for decision-making. Again, simply saying the likelihood is a 3 or a 5 won’t help executives. It’s not perfect, but it is better than saying the risk has a “moderate” chance of occurring.

Pivoting to impact, let’s say you retain your 1-5 (insignificant-severe) scale but add different parameters on what a 3 means or a 2 means. For an example, your company relies heavily on computer systems to operate. A severe impact could consist of an “outage affecting > 25% of customers or unavailable for > 5 days” while an insignificant impact could be where an outage affects “…<1% of customers or Unavailable <4 hours.”

Of course, full quantitative methods where you have hard data to run models provides better insights on how a particular strategy, risk mitigation, or other idea could work out.

But according to Douglas Hubbard’s book Failure of Risk Management: Why It’s Broken and How to Fix It and the presentation mentioned earlier, there are several roadblocks to pursuing full quantitative assessment, with the most common being a lack of experience in statistics and modeling.

If that’s the situation you find your company in, a hybrid approach like the one described above can be a good first step for dipping your toe in the quantitative waters.

Regardless of where you’re starting out, it’s important that you’re always seeking to improve assessment processes since uncertainty will only continue to grow in the years ahead.

What have you tried doing to move your company from a qualitative to a more quantitative risk assessment approach?

I’m always interested in hearing more about what other risk practitioners are doing to help their organizations improve risk management processes. If you’re able to share your experience, do so by leaving a comment below or join the conversation on LinkedIn.

If your company is struggling to move beyond a basic qualitative approach or has found a strict quantitative approach too overwhelming, please schedule a call so SDS can help you move in the right direction.

Featured image courtesy of  Airam Dato-On via Pexels.com

Posted in

Sign Up For Our Newsletter

Sign Up For Our Newsletter

SDS-Logo
about-sidebar-v2

Meet Carol

Helping companies achieve their vision and strategy, and succeeding in today's turbulent world, is something I'm honored to be a part of. Whether you're an occasional blog visitor or a long-term client, thank you for letting us be a part of your journey.

Most Recent Posts

The 12 Days of ERM Christmas

Without a doubt, one of my family’s favorite holidays is Christmas. Part of the fun, especially for our son, is seeing what “Santa” brought, but most importantly, we treasure the spirit of peace and goodwill the season brings. And after what seemed to be a never-ending warm spell, the weather is expected to be good…

Read More

Don’t Let Goals and Initiatives Be Blindsided by External Events

As the end of the year draws near, I think we’d all agree that while it wasn’t without its challenges, this year also wasn’t quite as turbulent as the previous two. While a lot of people are juggling company parties, shopping for friends and family, and special activities for the kids, most companies are putting…

Read More

Going the Distance: Ensuring Successful Execution of Strategic and Annual Initiatives

Strategic planning is a challenge – of all people, I understand… After all the meetings, risk and data analysis, and brainstorming of the preceding months, it’s tempting to think this is the end of the road and you can relax. Contrary to this common perception though, this is exactly not the time to relax, but…

Read More

Avoid Rookie Mistakes and Protect your Internal Reputation

Be honest – have you ever done something that you soon realized was a real rookie mistake? Me raising my hand… Considering the nature of ERM’s role to ask questions and challenge assumptions (often during conversations with executives), it can be argued that, in at least some cases, the expectations bar for risk professionals is…

Read More

ERM at Thanksgiving – An Illustration of Risk Management in Action

On occasion, I like to take some of the concepts we risk professionals think about in our jobs and apply them to different personal situations…take some of the same concepts we use when working with executives to develop corporate strategy and manage risks or uncertainty around that strategy. It’s Thanksgiving week in the U.S. –…

Read More

Why Quantitative Risk Assessment is Not Just the Best But the Only Option – A Conversation

Periodically, I have the pleasure of speaking one-on-one with Hans Læssøe on a variety of topics around ERM, strategic risk, and other issues and trends. As you know from my previous conversations (here, here) and posts featuring his work, Hans was formerly a practitioner at the iconic LEGO Company, but even more notably, is a…

Read More

The Three Lines Model – 3 Reasons Why I Don’t Like It

Everyone likes a clear-cut template that offers an easy way to create or manage something…I mean what’s not to like about a step-by-step process for accomplishing what you want? Sometimes this can work without any issues, such as the case with the Project Management Book of Knowledge (PMBOK), ISO 9001 standard, or a new cooking…

Read More

5 Avenues for Expanding your ERM Knowledge

One thing I was taught to appreciate from a young age was the value of education and knowledge. It didn’t necessarily matter what the subject was, just that I always maintain a learning or growth mindset regardless of my current status in life. This mindset has served me well over the years, and it’s a…

Read More

Storytelling and Risk Management – Developing Skills that Technology Cannot Replace

It’s amazing how technology has developed and changed our working world over time. Imagine trying to run my risk and strategy consulting firm without tools like Zoom, Box, Slack, and other ERM-specific technology tools. There is no way we would be able to serve our clients the way that we do. Just consider how the…

Read More

3 Phases to Creating and Launching an ERM Program Focused on Organizational Success

If you’ve been handed the task of creating an ERM program for your organization, let me first offer my congratulations quickly followed by my empathy for the task ahead of you. I don’t say that to scare you but to provide a small dose of reality. Building, launching, and refining an ERM program that is…

Read More