Early last month, I published an article explaining why I don’t discuss quantitative assessment methods like modeling, Monte Carlo simulation, and others very much and what companies must have in place before they can use methods like this to guide their decision making and strategic planning.
In short, without establishing corporate governance policies, a strategic plan, or clearly defined roles and responsibilities, any sort of advanced ERM methods will fail to be a useful tool for ensuring a company is taking the right amount of risks in pursuit of strategic objectives.
In this article on enterprise risk assessment from two years ago, I explain how qualitative methods use descriptive elements to rank a particular risk. Risk managers can then take this information and develop risk scores based on likelihood, impact and other parameters. In some cases, these scores can be plotted on a heat map, which as COSO explains, is “…a graphical representation of likelihood and impact of one or more risks.”
However, thought leaders at the forefront of ERM are extremely skeptical about the value qualitative risk assessments and heat maps can provide an organization. For example, in response to my article from last month, Hans Læssøe comments:
Relying on qualitative risk assessment is infected by human biases and potentially dangerous…it will not help you run [or] perform better, so waste as little effort on this as possible.
If a company’s goal with ERM is to make better decisions and improve performance, Hans explains:
…there is no avoidance of quantitative approaches – including Monte Carlo simulation. This will require skilled people and effort – but also add the value of enhancing performance.
Other thought leaders are even more critical of qualitative methods, including David Vose, who explains in this article:
Qualitative risk analysis may be appealingly simple, and lots of companies use it, but it’s almost valueless for making risk-based decisions.
David goes onto explain in his article and in a presentation at Risk Awareness Week 2019 how qualitative scores are “…too imprecise and inflexible.”
If you browse around my site and others like RIMS, NC State’s ERM Initiative, and others on the topic of risk assessments, you will find a lot of information on qualitative methods, but…
Based on comments from Hans, David, and others, should we assume qualitative risk assessments are fatally flawed?
I certainly agree with Hans and David that qualitative methods, especially how they’re currently done by many companies, are fatally flawed due to human bias.
While I agree with their opinions, the actual move from qualitative-based risk assessments and heat maps to quantitative modeling is an entirely different matter.
For many companies, qualitative risk assessment is all they know…asking them to go from this to more quantitative methods like modeling and Monte Carlo simulation represents a huge shift. If the culture of the organization will not support such a change, it is doomed to fail.
As an example, heat maps are quite popular with not only the general risk management community, but executives and boards as well. This popularity is likely due to their simple color scheme (red/yellow/green) and quick glance comprehension – finding or developing a quantitative reporting style that meets these needs will be challenging.
In the meantime, companies can use different techniques to overcome bias and quantify risks that are normally qualitatively assessed…
Despite the urgency for companies to take informed risks in pursuit of strategic objectives, getting to the ideal situation promoted by Hans and others will take a long time. Certain companies like those in finance may already use quantitative methods, but for many, this is simply out of reach at this time.
However, this doesn’t mean companies shouldn’t be on the lookout for ways to address the shortcomings of qualitative risk assessments.
One way to overcome bias is to ask executives what history tells them. They may consider a risk to be of grave importance, but after careful examination, you may learn that when the risk materialized in the past, it was not that significant after all.
There are numerous books and resources out there for helping reduce bias.
Many people think a quantitative risk assessment is providing a scale of 1 to 5 with criteria associated with each number. No. But taking a risk that is normally thought of as qualitative or difficult to put a number to it…that is the hard part.
Let’s walk through an example using a frequently discussed risk: the ability to recruit and retain quality talent.
Rather than say that the risk is a 4 on a scale of 1 to 5, there needs to be some hard numbers put to it. If your organization cannot recruit and retain quality talent, what will the organization not be able to achieve? Will certain revenue numbers not be met? Will products be designed and manufactured on time? Putting scope and boundaries on a risk will help you identify those numbers. And when all else fails, you can always check out the book “How to Measure Anything” by Douglas Hubbard.
While Hans, David Vose, and others are correct about the shortcomings of qualitative risk assessments and are visionary in how risk management must change in order to help companies create a strategic advantage, getting there is going to be a long road for many.
But as Hans also explains, the world is changing at such an unprecedented speed that companies may not have a choice if they wish to remain competitive.
Do you think qualitative risk assessments are fatally flawed?
What methods have you used to overcome bias in your organization when it comes to risk management and decision-making?
Share your thoughts by leaving a comment below or join the conversation on LinkedIn.
And if you are struggling to develop risk assessment methods for your company, please don’t hesitate to send me an email to discuss your specific situation today.
Featured image courtesy of Kevin Ku via Unsplash.com
