The evolution of the model risk management processes

Model risk is a relatively new area which only recently evolved as a separate area from operational risk. The work published by the Federal Reserve first and the European Baking Authority and Bank of England later has helped to establish the foundations for a model risk framework. The real challenge for model risk is to evolve from a pure cost to a value-added area for the financial institutions. This journey requires the application of many elements including a consistent model definition and performance monitoring of model risk in line with the risk appetite and business model of the financial institutions. The final step is to identify a coherent and clear way to aggregate and report model risk to allow the senior management to fully understand it in detail and integrate it in the broader risk management strategy.

Despite the fact that model risk has been elevated to the standard of a separate area in risk management only recently, in the financial history we had many incidents due to poor management of model risk with associated huge losses. Here are some examples:

Tacoma Narrow Bridge (1940)

It was a suspension bridge that opened on the 1^st of July 1940 and collapsed on the 7^th Nov 1940. The bridge collapsed because the vertical movements of the wind represented a weakness of the deflection theory used to design the bridge. Unfortunately, the engineers were not aware of it and so they did not take this into consideration when they validated their calculations¹.

Bank of America (2014)

They made a mistake in their Comprehensive Capital Analysis and Review (CCAR) because they applied an incorrect adjustment to the fair value option to certain structured notes resulting in an overstatement of regulatory capital amounts and ratios. This data/process error caused $4B reduction in reported capital².

London Whale (2012)

A quant of the London office developed a new VaR (Value at Risk) model for the Chief Investment Office (CIO). The model operated through a series of Excel spreadsheets, which had to be completed manually. Unfortunately, the validation of the model did not capture the fundamental flaws of the model with a consequence of a trading losses of $5.8B³.

Model risk is extremely challenging to measure and mitigate because of the intrinsic nature of the models.

​​​​​​​Not to mention the recent financial crisis, where most of the models, used to evaluate the price and risk embedded in the structured products like securitisations, showed their limits in terms of accuracy.

While the Federal reserve (FED) guidelines on model risk represent a valid support⁴, model risk is extremely challenging to measure and mitigate because of the intrinsic nature of the models: they are proxies based on assumptions to evaluate and predict real variables.

Having said that, it is the author’s view that model risk is on evolutionary process from a pure model validation to a risk management tool which provides value-added for the financial institutions (model risk management, MRM)⁵. This is an important point given the common confusion between these two concepts. Model validation is primarily driven by regulatory requirements while the MRM is the approach a financial institution uses to assess, mitigate and monitor model risk in line with their risk appetite.

A last introductory note, the reader can come across some concepts more than once. This is partially linked to the nature of the topic and intentional due to the educational scope of this article.

The evolution of model risk management (MRM) process

The evolution of MRM can be represented in three key phases as shown in Figure 1.

In relation to phase 1, each organisation usually deals with the more difficult and fundamental point: the model definition. There are different approaches to define a model. The more common is the one provided by the FED⁴. Of course, it is necessary to add some specific features to take into consideration the risk sources and the associated consequences⁶.

At the time of this article, most of the challenger banks are between phase 1 and phase 2 while the tier 1 banks are fully in phase 2 and touching some of the features of phase 3 in terms of the standards for model development and validation.

Read the rest of this paper here:

References

1. ​​​​​​​“Tacoma narrows bridge” available at https://www.wsdot.wa.gov/TNBhistory/Machine/machine3.htm, last accessed on the 7th October 2019.
2. “BofA Forced To Suspend 2014 Capital Plan Due To Error In Ratio Calculations” available at https://dealbook.nytimes.com/2014/04/28/bank-of-america-suspends-buyback-and-dividend-increase/, last accessed on the 7th October 2019.
3. “How The London Whale Debacle Is Partly The Result Of An Error Using Excel” available at https://www.businessinsider.com/excel-partly-to-blame-for-trading-loss-2013-2?r=US&IR=T, last accessed on the 7th October 2019.
4. “Supervisory guidance on model risk management” available at https://www.federalreserve.gov/supervisionreg/srletters/sr1107a1.pdf, last accessed on the 7th October 2019.
5. “Model validation vs model risk management”, available at https://www.theirm.org/media/3497273/IRM_Model-Risk-Management_FINAL.pdf, last accessed on the 7th October 2019.
6. “Definition of model risk”, available at https://assets.kpmg.com/content/dam/kpmg/pdf/2016/06/KPMG-Whitepaper-Model-Risk-Management-2016.pdf, last accessed on the 7th October 2019.