Implementing Model Risk Management Principles (SS1/23)
Overview
Banks are heavily dependent on models to support business decisions and risk analysis. These models are becoming more sophisticated and complex. With businesses starting to use artificial intelligence (AI) and machine learning (ML), models are likely to become even more complex and change more rapidly.
On 17 May 2023, the Prudential Regulation Authority (PRA) published its supervisory statement (SS) on model risk management (MRM) which incorporated the feedback it received to its proposal issued in June 2022 (see CP6/22). The PRA set out five key MRM principles that come into effect on 17 May 2024.
To mitigate model risks, the PRA expects banks to embed MRM into their culture, processes and governance.
“The purpose of this SS is to support firms to strengthen their policies, procedures, and practices to identify, manage, and control the risks associated with the use of all models, developed in-house or externally, including vendor models, and models used for financial reporting purposes.” SS1/23 – Paragraph 1.3
Banks are expected to conduct an initial self-assessment of their pre-existing MRM framework against the five MRM principles and, where appropriate, prepare remediation plans to address any identified shortcomings. Ongoing review and further self-assessments should be carried out at least annually.
Scope and Applicability
The MRM principles cover the model lifecycle and apply to all types of models used to inform key business decisions – whether developed in-house or externally (including vendor models). The PRA views model risk as a separate identifiable business risk, which banks should recognise and mitigate by embedding the MRM principles - including strong governance.
The PRA states that the SS is to be applied by banks who have Internal Model (IM) approval to calculate regulatory capital requirements[1]. Notwithstanding, all banks will find the principles useful.
“SS is relevant to all regulated United Kingdom (UK)-incorporated banks…with internal model approval to calculate regulatory capital requirements….However, the PRA considers that those firms may find the proposed principles useful, and are welcome to consider them to manage model risk within their firm” SS1/23 – Paragraph 1.2
The application of MRM adopted by banks should be proportional to the size and complexity of the bank and its models.
[1] IM approval to calculate capital requirements for credit risk, market risk or counterparty credit risk
Why is Model Risk Management (MRM) Important for All Banks?
Prevalence of models
Banks’ increasing reliance on models and scenario analysis to assess risks, as well as the evolution of sophisticated modelling techniques, highlights the need for sound model governance and effective MRM practices. Inadequate or flawed design and implementation, or inappropriate use of models, could lead to adverse consequences including a deterioration in a bank’s prudential position, non-compliance with applicable laws or regulations, or damage to a bank’s reputation.
Speed of change
With the rapidly changing environment, digital landscapes, and evolution of more sophisticated modelling techniques, banks’ use of models will continue to increase and become more complex as new model types are introduced. Examples include the quantification of the financial risks associated with climate change and the introduction of AI and ML techniques.
Perceived quality of model governance frameworks
Poor quality model submissions, issues with IFRS9,[2] and a general reluctance to invest and bolster MRM functions (as evidenced by recent PRA reviews) have highlighted the increasing need for sound model governance and effective MRM practices.
[2] Internal Financial Reporting Standards (IFRS) 9 – Financial Instruments
What is a Model?
The PRA defines a model as:
“…a quantitative method, system, or approach that applies statistical, economic, financial, or mathematical theories, techniques, and assumptions to process input data into output. The definition of a model includes input data that are quantitative and / or qualitative in nature or expert judgement-based, and output that are quantitative or qualitative.” SS1/23 – Principle 1.1 (a) – Model definition
The diagram below schematically represents the various elements of a model. The inputs and outputs can be quantitative and/or qualitative in nature. The inputs (or assumptions) can even be based on expert-judgement.
The output generated by the model is used to take business and other decisions that ensures franchise viability and safety of the bank, including areas such as:
Overall strategy;
Risk management;
Financial management;
Capital management;
Liquidity management;
Operational activities (e.g., anti-money laundering); and,
Product pricing.
The models can range from sophisticated systems to Excel-based end-user computing (EUC) applications. Based on the model definition, the bank must determine which of its systems, applications, methods, approaches or processes should be identified as models, and then apply the appropriate MRM Principles to these models.
A few examples of systems, processes or methods commonly used in small- and medium-sized banks that can be classified as models are listed below:
Expected Credit Loss (ECL) model used to determine IFRS9 provisions.
Internal credit rating model.
Capital planning models used in 3- to 5-year business plans.
Capital stress testing model (used as key element in the Internal Capital Adequacy Assessment Process exercise).
Liquidity stress testing model (used as key element in the Internal Liquidity Adequacy Assessment Process exercise).
Interest Rate Risk in the Banking Book (IRRBB) models to evaluate economic value of equity and net interest income measures.
Capital adequacy model (including determining capital available; and capital requirements for credit, market, operational, counterparty credit and CVA risks).
Funds Transfer Pricing model.
Risk Adjusted Return on Capital (RAROC) model.
Intra-day liquidity management model.
Daily liquidity buffer management/forecasting model.
Market risk PV01 model, or VaR model for investment portfolios.
The MRM Principles
The board of directors and senior management of banks are ultimately responsible for establishing a sound MRM framework that embeds the five principles, which are shown in the graphic below.
Principle 1 – Model identification and model risk classification
Banks should have an established definition of a model that sets the scope for MRM, a model inventory[3] and a risk-based tiering approach[4] to categorise models to help identify and manage model risk. The definition to be used is set out in the SS is outlined in the previous section (What is a Model?).
Key action items:
1. Adopt a definition for a model.
2. Prepare a bank-wide model inventory (including external and third-party vendor models).
3. Categorise the models identified in the model inventory into different risk tiers (e.g., High, Medium, Low).
[3] In addition to the basic information about the model like name of the model, business owner, department(s) using the model, current model status; the model inventory should also capture the purpose and use of the model, assumptions and limitations, findings from validations, governance details, and inter-dependencies.
[4] For small- and medium-sized banks, this can be as simple as categorising it as low, medium or high risk before applying any management/mitigating actions; based on quantitative and qualitative factors – including quality of data and the potential impact on the business.
Principle 2 – Governance
Banks should have strong governance oversight with a board that promotes an MRM culture from the top through setting a clear model risk appetite. The board should approve the MRM policy and appoint an accountable individual to assume the responsibility to implement a sound MRM framework that will ensure effective MRM practices. Banks should ensure the responsibilities are part of relevant Senior Management Function (SMF)[5] of the Senior Managers Regime (SMR) Statement of Resonsibilities.
Key action items:
1) Board - establish a bank-wide MRM framework and policy (reviewed at least annually).
2) Board – set model risk appetite (reviewed at least annually).
3) SMF responsible – implement processes and procedure to embed the MRM framework and policies (including external and third-party vendor models).
4) Internal Audit – assess the effectiveness of the MRM framework (at least annually).
[5] For small- and medium-sized banks this is most likely to the SMF4 function (Chief Risk)
Principle 3 – Model development, implementation, and use
Banks should have a robust model development process with standards for model design and implementation, model selection, and model performance measurement. Testing of data, model construct, assumptions, and model outcomes should be performed regularly in order to identify, monitor, record, and remediate model limitations and weaknesses. Model development documentation should be sufficiently detailed so that an independent third party with the relevant expertise would be able to understand how the model operates, to identify the key model assumptions and limitations, and to replicate any parameter estimation.
An example model life cycle is shown in the diagram below.
Key action items:
For each existing model and new models, document the following elements to ensure that an independent third-party with suitable expertise can understand how the model works:
a. the purpose;
b. design principles;
c. calculation methodologies;
d. use of data;
e. assumptions (including expert judgement) and limitations;
f. parameters (including interfaces);
g. testing approach (including validation and approval process);
h. model adjustments;
i. use of the output results and calibrations; and.
j. IT infrastructure and environment (including supporting systems).
Note: The high-level development, implementation and use methodology is to be documented in the MRM Policy, or alternatively it can also be documented separately based on the bank’s specific preferred approach.
Principle 4 – Independent model validation
Banks should have a validation process that provides ongoing, independent, and effective challenge to model development and use. This forms part of the model life cycle shown above. The individual or body within a bank responsible for the approval of a model should ensure that validation recommendations for remediation or redevelopment are actioned so that models are suitable for their intended purpose.
Key action items:
1) Independently validate any changes to an existing model or when a new model is introduced.
2) On a regular basis re-validate the models listed in the model inventory, to ensure that they meet the overall objectives. The periodicity can set based on the model risk tier (e.g., for small-and medium-sized banks High-risk on an annual basis, and Medium-risk on a bi-annual basis and Low-risk once every 3 years).
Principle 5 – Model risk mitigants
Banks should have established policies and procedures for the use of model risk mitigants when models are under-performing and should have procedures for the independent review of post-model adjustments (PMA).
Key action items:
1) Document all post-model adjustments that are made for each model (if not already included in the model development documentation) and document the process as to how the PMA is applied and validated.
2) Performance of the model (e.g., variance) is evaluated on a regular basis to ensure that the risk mitigants (e.g., PMA) applied is in line with the model objectives.
Note: The approval process for PMA and any other risk mitigants are to be documented in the Model Risk Policy.
Other Considerations
Financial reporting and external auditors
The PRA expects banks to report on the effectiveness of MRM for financial reporting including to the audit committee at least annually. To facilitate effective audit planning, the PRA expects banks to ensure that this report is available on a timely basis to inform their external auditor’s assessment of, and response to, the risk of material misstatement as part of the statutory audit. For small- and medium-bank, this might be limited to the ECL calculations done as part of the bank’s IFRS9 model.
Proportionality
The MRM Principles represent core risk management practices for all models and all risk types. The practical application of the principles by all banks should be commensurate with their size, business activities, and the complexity and extent of their model use. For example, for banks with a smaller number of models or less complex models, maintaining a model inventory should be less burdensome, and the criteria for classifying models into tiers can be materially simpler than for banks with a wider range of models or more complex models.
Externally developed models, third-party vendor products
Boards and senior management are ultimately responsible for the management of model risk, even when there are outsourcing or third-party arrangements. Banks should:
· satisfy themselves that the vendor models have been validated to the same standards as their own internal MRM expectations;
· verify the relevance of vendor supplied data and their assumptions; and,
· validate their own use of vendor products and conduct ongoing monitoring and outcomes analysis of vendor model performance using their own outcomes.