In my years as both a risk practitioner and ERM consultant, there has and continues to be intense debate around methods for assessing risks. (If you are unfamiliar with the meaning of qualitative and quantitative, this article provides a quick overview.)
To illustrate this debate, take the following two comments from a previous article asking whether quantitative is the only future of risk management:
As you can see from just these two comments on a blog post, it’s interesting the different perspectives you will run into, and like so many issues in today’s world, some of comments I’ve come across can be a little on the mean side. (Why can’t people provide constructive criticism without being mean?)
Qualitative methods can be easier to implement and maintain, especially for companies without strong modeling and statistical analysis capabilities. As I’ve seen over and over, qualitative risk assessment is an option for many organizations starting out simply because they will be overwhelmed if they jump into a full quantitative approach.
This overwhelm can be a huge setback for any risk management initiative that can possibly take years to dig out from…
But as promoters of a quantitative approach contend, qualitative-based risk assessment can be fraught with biases and even be dangerous since you are relying on subjective impulses of the person(s) giving the score as opposed to objective numbers, or as psychologist and Nobel prize winner Daniel Kahneman explains:
Overconfident professionals sincerely believe they have expertise, act as experts and look like experts. You will have to struggle to remind yourself that they may be in the grip of an illusion.
Of course, I’m not going to argue too much about the shortcomings of qualitative assessments, especially when you throw heat maps and risk matrices into the mix, because it is true – there are certainly downsides to using the method, some of which I have personally experienced. Honestly, I haven’t attempted to use a heat map since the early days of my ERM career, because as Douglas Hubbard says in this presentation, tools like this provide no clear answer to the question “Should we spend $X to reduce risk Y or $A to reduce risk B?”
But when I speak with companies, it becomes clear how most of them simply aren’t equipped to take full advantage of quantitative risk assessment methods, modeling, and so on. Fortunately, I have found that…
There’s a middle way to obtain probability ranges without extensive statistics experience.
A strict qualitative assessment method that relies on a 1-5 or low-medium-high scale is certainly fraught with all kinds of pitfalls – I get it.
However, as I’ve explained in other posts, sometimes this is the best you’ll be able to do, especially in the beginning stages of any ERM initiatives. In this case, you do what you can, knowing and communicating that the method is not permanent. As the program matures and capabilities grow, you and executives will need to move into more sophisticated assessment methods.
One approach that I use with some of my clients (…many of whom fall into this situation) is to assign specific criteria behind the scale/rating.
You see, one of the criticisms of qualitative (which I echo) is that the labels of “low”, “medium”, and “high” can mean different things to different people.
However, when you set probability ranges for each rating levels on the scale, or specific attributes behind what a level 3 impact means, you give the executive, manager, or information user a clear idea of what a specific ranking means in the context of a specific risk.
For example, when surveying the likelihood of risk(s) for a client, having probability ranges of <10%, 10-40%, 40-60%, 60-90%, and >90% can provide more clarity for decision-making. Again, simply saying the likelihood is a 3 or a 5 won’t help executives. It’s not perfect, but it is better than saying the risk has a “moderate” chance of occurring.
Pivoting to impact, let’s say you retain your 1-5 (insignificant-severe) scale but add different parameters on what a 3 means or a 2 means. For an example, your company relies heavily on computer systems to operate. A severe impact could consist of an “outage affecting > 25% of customers or unavailable for > 5 days” while an insignificant impact could be where an outage affects “…<1% of customers or Unavailable <4 hours.”
Of course, full quantitative methods where you have hard data to run models provides better insights on how a particular strategy, risk mitigation, or other idea could work out.
But according to Douglas Hubbard’s book Failure of Risk Management: Why It’s Broken and How to Fix It and the presentation mentioned earlier, there are several roadblocks to pursuing full quantitative assessment, with the most common being a lack of experience in statistics and modeling.
If that’s the situation you find your company in, a hybrid approach like the one described above can be a good first step for dipping your toe in the quantitative waters.
Regardless of where you’re starting out, it’s important that you’re always seeking to improve assessment processes since uncertainty will only continue to grow in the years ahead.
What have you tried doing to move your company from a qualitative to a more quantitative risk assessment approach?
I’m always interested in hearing more about what other risk practitioners are doing to help their organizations improve risk management processes. If you’re able to share your experience, do so by leaving a comment below or join the conversation on LinkedIn.
If your company is struggling to move beyond a basic qualitative approach or has found a strict quantitative approach too overwhelming, please schedule a call so SDS can help you move in the right direction.
Featured image courtesy of Airam Dato-On via Pexels.com
Sign Up For Our Newsletter
Sign Up For Our Newsletter
Helping companies achieve their vision and strategy, and succeeding in today's turbulent world, is something I'm honored to be a part of. Whether you're an occasional blog visitor or a long-term client, thank you for letting us be a part of your journey.
Most Recent Posts
Without a doubt, one of my family’s favorite holidays is Christmas. Part of the fun, especially for our son, is seeing what “Santa” brought, but most importantly, we treasure the spirit of peace and goodwill the season brings. And after what seemed to be a never-ending warm spell, the weather is expected to be good…Read More
As the end of the year draws near, I think we’d all agree that while it wasn’t without its challenges, this year also wasn’t quite as turbulent as the previous two. While a lot of people are juggling company parties, shopping for friends and family, and special activities for the kids, most companies are putting…Read More
Strategic planning is a challenge – of all people, I understand… After all the meetings, risk and data analysis, and brainstorming of the preceding months, it’s tempting to think this is the end of the road and you can relax. Contrary to this common perception though, this is exactly not the time to relax, but…Read More
Be honest – have you ever done something that you soon realized was a real rookie mistake? Me raising my hand… Considering the nature of ERM’s role to ask questions and challenge assumptions (often during conversations with executives), it can be argued that, in at least some cases, the expectations bar for risk professionals is…Read More
On occasion, I like to take some of the concepts we risk professionals think about in our jobs and apply them to different personal situations…take some of the same concepts we use when working with executives to develop corporate strategy and manage risks or uncertainty around that strategy. It’s Thanksgiving week in the U.S. –…Read More
Periodically, I have the pleasure of speaking one-on-one with Hans Læssøe on a variety of topics around ERM, strategic risk, and other issues and trends. As you know from my previous conversations (here, here) and posts featuring his work, Hans was formerly a practitioner at the iconic LEGO Company, but even more notably, is a…Read More
Everyone likes a clear-cut template that offers an easy way to create or manage something…I mean what’s not to like about a step-by-step process for accomplishing what you want? Sometimes this can work without any issues, such as the case with the Project Management Book of Knowledge (PMBOK), ISO 9001 standard, or a new cooking…Read More
One thing I was taught to appreciate from a young age was the value of education and knowledge. It didn’t necessarily matter what the subject was, just that I always maintain a learning or growth mindset regardless of my current status in life. This mindset has served me well over the years, and it’s a…Read More
It’s amazing how technology has developed and changed our working world over time. Imagine trying to run my risk and strategy consulting firm without tools like Zoom, Box, Slack, and other ERM-specific technology tools. There is no way we would be able to serve our clients the way that we do. Just consider how the…Read More
If you’ve been handed the task of creating an ERM program for your organization, let me first offer my congratulations quickly followed by my empathy for the task ahead of you. I don’t say that to scare you but to provide a small dose of reality. Building, launching, and refining an ERM program that is…Read More