Boosting regulatory assessment with generative AI: Prometeia’s approach for credit risk models

Generative AI (GenAI) technologies are transforming multiple industries by automating labour-intensive tasks leveraging on their ability to process vast amounts of data and text. In the financial sector, GenAI applications are diverse, ranging from customer service optimisation to investment analysis. By investing in these tools, institutions can streamline complex processes, including the assessment of regulatory compliance of risk models. This article presents Prometeia’s approach for the automation of credit risk model validation using a combination of cutting-edge AI techniques.

The broader role of generative AI in the financial sector

Generative AI, with its potential, represents a natural and enhanced evolution of predictive AI, building on and advancing its successful applications. It has transformed multiple aspects of the financial services industry. Customer service has significantly benefited from AI-driven tools, providing 24/7 tailored solutions based on historical customer data, enabling real-time issue resolution. Fraud detection has also been enhanced, with GenAI excelling at analysing vast amounts of unstructured data to identify subtle patterns that traditional methods may overlook. In market analysis and prediction, AI-driven sentiment analysis of news articles and social media posts allows institutions to anticipate market shifts and make informed decisions. However, one of the most impactful applications of generative AI lies in regulatory compliance, where automated solutions are increasingly necessary to handle the growing complexity of financial regulations.

Automating the assessment of credit risk models: A new frontier for Generative AI

Ensuring that banks’ internal risk models comply with regulatory guidelines is critical to maintaining the stability of the financial system. Validators, auditors, model developers and supervisors are responsible for ensuring and validating the compliance of these models. Traditionally, this has been a manual, time-consuming process involving the review of long and complex documents. By incorporating generative AI solutions, institutions can significantly reduce the time and effort involved, while freeing up valuable resources for higher value-added tasks. Moreover, for such stakeholders it is crucial to ensure homogenous standards for the output of the assessment process, particularly if performed by teams of professionals with different backgrounds and levels of experience. Generative AI-based technologies can help to standardise the compliance assessment procedures, thereby enhancing the coherence of the corresponding outcomes.

At Prometeia, we have developed a comprehensive tool designed to automate the validation of credit risk models. Our solution analyses key components, such as the bank’s methodological framework, risk parameters and judgmental decisions. These elements are then compared with the relevant regulatory as well as bank’s internal guidelines to assess the compliance of the bank’s risk models. A chatbot is also available to allow the user to deep-dive into specific aspects of the model.

How the tool works: Key features and technical insights

At Prometeia, we go beyond simply integrating LLMs. Our solution leverages a sophisticated combination of technologies to ensure both accuracy and efficiency in automating credit risk model assessment:

1. Document ingestion and intelligent pre-processing: The tool ingests and processes both regulatory and internal documents, ensuring that all relevant information is captured in a format suitable for analysis. The pre-processing techniques applied are able to understand the semantic and structural hierarchy of the processed document.
2. Text splitting: To handle complex, large-scale documents effectively, our tool divides them into manageable, self-contained chunks. Each chunk is designed to retain enough contextual information to be meaningful while being concise enough to facilitate efficient processing.
3. Advanced retrieval-augmented generation (RAG): By applying customized RAG techniques based on semantic similarity metrics, the tool retrieves only the most pertinent sections of the documents for further analysis. This ensures that the tool focuses its resources on the most relevant content, streamlining the validation process and avoiding hallucinations.
4. Large language model: The selected text chunks are then processed by a state-of-the-art LLM, currently GPT-4o from Azure OpenAI. The LLM compares the retrieved sections of the bank’s documentation with regulatory requirements to determine compliance and generates an output that emphasises both the model’s strengths and its limitations. Our system’s design is flexible and scalable, enabling us to switch to alternative LLMs, such as open-source models like Llama or even fine-tuned versions of it. Moreover, the confidentiality of the ingested documents is safeguarded, in line with all the relevant EU Regulations.
5. Multi-step processing and contextual retrieval: Throughout the validation process, the tool employs contextual retrieval techniques to enhance document retrieval accuracy. Furthermore, prompts are iteratively refined by incorporating feedback and information extracted from previous steps, enhancing the tool's comprehension of the documents and directing it through the validation process. This multi-step approach guarantees a thorough assessment, addressing every nuance in the compliance check. The iterative feedback loop ensures that no critical detail is overlooked, resulting in highly accurate and comprehensive validation.

Figure 1 – Schematic illustration of the functioning of Prometeia’s GenAI assistant for model assessment

Looking forward: Expanding capabilities

Although our current focus is on Pillar 1 credit risk models, our tool is designed to be adaptable and scalable. We are currently expanding its capabilities to cover other geographies and regulatory frameworks, such as IFRS9, IRRBB and additional risk categories, including cyber risk, operational and market risks. The scalability and flexibility of our solution will enable banks to manage evolving regulatory demands across a wide range of domains. This adaptability is integral to the evolving regulatory landscape, as financial institutions face increasing scrutiny from global regulatory bodies.

Conclusion: The future of generative AI in risk management

At Prometeia, we believe the future of risk management lies in the seamless integration of advanced AI technologies. As financial regulations continue to grow in complexity, automating model validation processes will be essential for institutions to remain competitive and compliant. By investing in AI-driven solutions, financial institutions can not only free up valuable resources but also focus on more strategic initiatives, positioning themselves for long-term success in an increasingly complex regulatory landscape.

By leveraging the latest generative AI technologies and integrating state-of-the-art methodologies, Prometeia is at the forefront of creating tools that not only perform but excel in delivering precise, scalable, and intelligent solutions for regulatory compliance.

Join the Prometeia team at RiskMinds International and discuss the latest in credit risk management innovations.

References:

1. Peng, C., He, H., Liu, Z., Zhang, L., Cao, X., Khan, A.T., Katsikis, V.N., & Li, S. (2024). Empowering financial futures: Large language models in the modern financial landscape. EAI Endorsed Transactions on AI and Robotics.
2. OpenAI. (2023). GPT-4 Technical Report.
3. Bsharat, S.M., Myrzakhan, A., & Shen, Z. (2023). Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4.
4. Lewis, Patrick, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler et al. (2020) "Retrieval-augmented generation for knowledge-intensive nlp tasks." Advances in Neural Information Processing Systems
5. Llama Team, AI @ Meta (2024). The Llama 3 Herd of Models.