Steve Lindo, Course Designer and Instructor at Columbia University (MSc in Enterprise Risk Management) discusses how to bring new risk analytic techniques from the intelligence sector into financial risk management.
Three Established Approachs to Risk Analysis
In most business situations, one, or a combination of three risk analysis approaches, can be employed to good effect. The first approach is quantitative analysis using empirical data. This involves extracting information from observed data, using mathematical methods. Examples of this are determining the riskiness of a book of loans or property insurance, by calculating the frequency and severity of historical losses. The second approach is quantitative analysis using expert-generated data. Examples of this are risk analysis based on stress-tests, whose values are created by forecasting models, or credit ratings, which are calculated using criteria and models developed by experts. The third approach is qualitative analysis using critical thinking, based on domain experience, historical study, observation of trends and deductive reasoning. This is the default approach used when data is scarce or missing.
Strategic Risk Management – Conspicuous Analytical Failures
In one critical risk area, namely strategic risk, these three approaches have been shown to often fall short. You don’t have to look far for examples of this. In the financial sector, the aggressive program of sales incentives developed and systematized by Wells Fargo did not sufficiently weigh the potential damage to employee conduct. At JPMorgan, outsize trading authority was given to its Chief Investment Office without assigning appropriate controls or risk resources. At AIG, its derivatives and stock lending units grew tremendously without properly evaluation of their vulnerability to extreme stress conditions. At Lehman Brothers, the executives maintained faith in the resilience of the US housing market and their bank being too-big-to-fail, in spite of signals to the contrary.
Despite the high stakes involved, strategic risk management is still error-prone in the business world.
Examples of disastrous strategic risk mismanagement in the non-financial sector are also prevalent. At Volkswagen, deceptive emissions software was introduced without sufficient evaluation of the risk to its reputation and the magnitude of fines that could be imposed. At Target, the chain of stores it opened in Canada went bankrupt after two years of insurmountable supply chain problems. Blockbuster video rental went out of business after failing to adapt its business strategy to the advent of Netflix, and Westinghouse placed a huge, unsuccessful bet when it acquired Toshiba’s worldwide nuclear power business.
These examples indicate that, despite the high stakes involved, strategic risk management is still error-prone in the business world. There is a common theme linking these strategic risk management blunders. From my own experience and research, I’ve identified three kinds of strategic risk management process failures which are commonly to blame:
1. High-stakes decision-making typically takes place infrequently, meaning that participants have few opportunities to hone their strategic risk management skills.
2. The large amounts of data, expert projections and qualitative risk analysis which are contributed by multiple departments, functions and stakeholders in such situations can be a huge challenge to reconcile in a very short time frame.
3. Organizational biases frequently interfere with the objectivity of strategic risk analysis. For example:
a. “Here’s what I (the CEO) want to do, speak up anybody who thinks it’s a bad idea”
b. “The accountants have already run the numbers, so all you have to do is vote yes or no”
c. “We’ve already run this by our lawyers and they’re fine with it”
d. “You can ask as many questions as you like, but we have to make a yes/no decision by the end of this meeting”
I expect that these examples resonate with many of the readers of this article.
Intelligence Risk Analytic Techniques
The strategic risk analysis challenges in the intelligence sector are broadly the same as those in the business world – high stakes, short time availability and lots of uncertainty. What’s different in the intelligence sector is that the practice of strategic risk analysis is much more continuous, because high-stakes situations occur all the time, and the margin for error is much slimmer, since intelligence situations typically involve not just money but also property, lives and global influence. By way of illustration, there have been some noteworthy intelligence successes, such as the Camp David Accord and the capture of Saddam Hussain. There have also been noteworthy failures, such as Pearl Harbor and Iraq’s non-existent weapons of mass destruction. Driven by the frequency and severity of these situations, the intelligence sector has, by force of practice and iterative testing, developed and adopted a set of analytical methods which specifically address these challenges, known as Structured Analytic Techniques or SAT’s. These methods fall into three main categories: Challenge, Imagination and Diagnostic. Challenge SAT’s are used to question conventional wisdom, Imagination SAT’s promote out-of-the-box thinking, and Diagnostic SAT’s impose rigor on the risk analysis process. Together, these techniques foster cross-functional collaboration, transparency and intellectual rigor, while isolating political interference, reinforcing hard evidence, exposing biases and strengthening the overall credibility of the analysis.
For the purposes of this article, I’m going to focus on the Diagnostic techniques, because strategic risk analysis process, as I mentioned earlier, is the area which I consider especially failure-prone in the business world.
Key Assumptions Check
The primary diagnostic technique used in the intelligence sector is the Key Assumptions Check, or KAC, which is typically performed by a cross-functional team of representatives from some or all of risk management, operations, legal, accounting, strategic planning, tax and IT, as well as the business unit or product line involved in the proposition.
The Key Assumptions Check consists of 8 steps:
Step 1: Precisely determine the strategic objective or objectives of the business proposition. This preliminary step is known in the intelligence sector as the Bottom Line Up Front, or BLUF. Careful questioning of the principal actors and key stakeholders in a high-stakes situation is essential, in order to obtain absolute clarity about the Bottom Line.
Step 2: List all the key assumptions upon which the proposition rests. Every situation is different, but it’s typical for there to be 8-10 assumptions which all stakeholders agree are key to its success.
Step 3: Select the five key assumptions which by consensus are the most critical. Focusing on a limited number of assumptions has proven to be a crucial tactic for the effectiveness of the Key Assumptions Check process.
Step 4: Assign a rating to each of the top five assumptions in each of the following areas:
a) Impact on the success of the proposition
b) Level of certainty that the assumption will prove correct
c) Level of confidence in the data and/or judgments supporting that certainty
d) Impact on other assumptions if this one proves incorrect, and
e) Overall criticality of the assumption to the success of the proposition.
A 1-10 rating scale is typically used, where 10 is the strongest.
Step 5: Back-test all the KAC ratings against the BLUF, to make sure that they’ve been framed and rated in strict accordance with the objectives clarified in Step 1.
Step 6: Focus on any and all low ratings, discuss why they are low and what if any actions can be taken to raise them within the available timeframe. A common mistake to avoid is summing the ratings to determine the relative strength of each assumption. This is not the purpose of the Key Assumptions Check, which is to identify and analyze individual weaknesses which could dramatically affect the outcome.
Step 7: Draw up a set of conclusions and recommendations about the strategic risks in the proposition, and possible actions which could mitigate them.
Step 8: Submit the recommendations to the executive team tasked with making the decision, along with a summary of the Key Assumption Check’s supporting documents.
Real-Life Example
To demonstrate the power of the KAC technique in practice, I’ve selected one of the case studies I mentioned earlier as an example – Target Canada.
For readers unfamiliar with the story, Target’s CEO agreed to acquire Zellers’ Canadian stores for US$1.8 billion in 2011. Two years later, with great fanfare, Target Canada opened 124 stores. Two years after that, due to constantly empty shelves, overflowing warehouses, bad publicity and US$ 941 million of losses, Target Canada ceased operations.
Given these weaknesses, the disastrous outcome, in hindsight, was predictable.
A rigorous Key Assumption Check, before the CEO signed the deal, would have identified unacceptably low ratings for 3 key assumptions relevant at the time, which were:
Key Assumption #1
Target’s brand alone would attract Canadian shoppers to the old Zellers locations in great numbers. In fact, there was scant evidence to support this assumption.
Key Assumption #2
Target’s legendary supply chain efficiency, from vendor to warehouse to store to point-of-sale, would keep its Canadian stores’ shelves and its warehouses constantly full. In fact, Target decided not to adapt its long-established, in-house inventory and supply-chain management system to operate in Canada. Instead, it installed a state-of-the-art vendor system which its IT experts and newly-hired Canadian employees all struggled, and ultimately failed, to master.
Key Assumption #3
Target’s “can-do” culture could surmount any obstacles standing in the way of fulfilling its Canadian ambitions. This deeply-held bias had not been empirically tested in any prior foreign expansion, and was not challenged by Target’s executive team at the time.
Given these weaknesses, the disastrous outcome, in hindsight, was predictable.
Developing an Intelligence Risk Analytic Capability
Implementing SAT capability is a bit like implementing ERM, in that its components are not rocket science, but have to be tailored to each company’s unique risk profile, organizational structure and culture. One approach is to study SAT materials which are in the public domain and give it a try. The downside to this approach is that it’s asking a lot to conduct a successful live trial with an unfamiliar process in a high-stakes situation. A better approach is to select some key employees, risk managers for example, to undergo training in SAT principles and practice, then socialize their learning in-house, with briefings, case studies and live trials. Ultimately, I expect SAT’s to take root primarily in companies which already promote transparency, inclusiveness and risk discipline, the same characteristics which provide a solid grounding for the implementation of ERM.
Steve Lindo is a financial risk manager with over 30 years’ experience managing risks in ALM, funding, banking and trading portfolios. His current role is Principal of SRL Advisory Services, an independent consulting firm specializing in risk governance, education and strategy, financial technology innovation, and expert witness in complex litigation. He is a Lecturer and Course Designer at Columbia University’s School of Professional Studies, teaching Financial Risk Management to MSc students in Enterprise Risk Management.
Hear first-hand the most pressing development on the CRO's agenda and join technical risk experts as they discuss the upcoming challenges and opportunities, at RiskMinds International.