Search
Search

 

 

 

  • Risk
  • Model Risk Management
  • Blog
  • Investing
  • Banking
  • Global Research and Risk Solutions
July 05, 2022

Making the grade

How banks could pass the PRA muster on model risk management expectations

by Rachna Maheshwari, Associate Director - Model Risk Management
CRISIL Global Research and Risk Solutions

 

Overview of the CP 6/22 model risk management principles


Recently, the British Prudential Regulation Authority (PRA) came out with a consultation paper (CP) which aims to get banks to prepare ahead for managing model risks. It lays out expectations from banks, based on its five guiding principles for effective MRM.

 

Based on the consultation paper, we think that most major banks will need to revise some policies and procedures to fall in line with the new guidance, even if they have well-established MRM frameworks.

 

The Supervisory Statement (SS), which sets out the expectations for regulated United Kingdom (UK)-incorporated banks, building societies, and PRA-designated investment firms around MRM practices, has the following major objectives:

 

Build robust MRM practices around artificial intelligence (AI)/machine learning (ML): Given the dynamic financial services landscape and rapid adoption of AI/ML and other innovations such as digitization, robotics and automation, the SS encourages firms to have robust practices around identification, management, reporting, monitoring and mitigation of model risks arising from the adoption of these technologies.

 

Involve the board and senior management: Active senior management and board involvement in firms’ MRM governance processes are key to robust and effective MRM practices.  Therefore, the PRA wants firms to identify and assign responsibilities for overall MRM to the relevant senior management function (SMF) as well as to ensure board oversight over MRM. 

 

Proportionate implementation: This refers to the guiding principle that application of MRM practices and governance should be proportionate to the number, complexity, and materiality of models. Typically, smaller and ‘simpler-regime’ firms would have fewer and less complex models. While they too should establish the model definition, keep an inventory, and classify models, they need only to focus on the limited and basic elements of model governance. 

 

Self-assess the MRM function: The SS expects firms to set up a recurring process of self-assessment and evaluation of MRM frameworks, policies, and processes. It requires them to remediate and report on identified gaps, thereby enhancing and consistently maintaining high MRM standards. 

 

 

   PRA’s guidance on firms’ self-assessment

 

  • The implementation date of this policy is set at 12 months following its publication.
  • By or before the implementation date, firms applying the SS are expected to conduct an initial self-assessment of their existing MRM frameworks and policies and prepare and implement remediation plans for any identified gaps.
  • Thereafter, it is recommended that self-assessment of MRM frameworks, policies and procedures should be conducted at least annually, and any identified gaps and associated remediations should be documented.
  • Firms that qualify as simpler-regime firms should complete an initial self-assessment and conduct subsequent assessments at lower frequency, as required.
  • The outcomes of self-assessments, including identified gaps and remediations should be made available to the PRA for review on request.

 

A strategic and holistic view of MRM: The SS encourages PRA-regulated entities to establish MRM frameworks, procedures, and practices, comprehensively covering all model types used to inform business decision-making, including in-house and vendor models, and all aspects of model life-cycle management. It aims at establishing greater coherence and consistency in MRM across UK firms with comparable MRM practices and measurable outcomes, especially regarding the practices around key aspects of MRM. 

 

 

   Key aspects of a holistic MRM strategy

 

  • Model identification, risk classification and inventory management
  • Effective model governance, including:
    • Well-defined MRM policies and procedures
    • Well-established MRM roles and responsibilities
    • Clearly defined scope and policies for involving the third line of defence and internal audit review
    • Policies and procedures regarding the use of externally developed models, third party and vendor products
    • Policies regarding SMF accountability of the MRM function
    • Responsibilities and oversight scope of the Board of Directors for the MRM function
  • Model development, implementation, and use, including requiring firms to have a robust model development process with established standards, policies, and procedures for:
    • Model purpose definition, selection, and specification
    • Model development and use of modelling data
    • Model implementation
    • Model adjustments and expert judgement
    • Model development documentation
    • Supporting systems
  • Independent model validation, including:
    • Independent validation function: Responsible for initial validation, periodic re-validation, and independent review of the models.
    • Model process verification: Policies for thorough verification of model inputs, outputs, and calculations, including system implementation, processing, and user development applications.
    • Independent review: Established process for independent review of models including model components, model inputs, calculations and reporting outputs, conceptual soundness, critical analysis of the model development process, evaluation of qualitative information and judgement used for model development, and additional testing and analysis as required.
    • Model monitoring: A framework that frequently tests modelling data, model construct, assumptions, and model outcomes, performed to identify, monitor, record, and remediate model limitations and weaknesses.
    • Model risk mitigants when models under-perform: Well-defined and consistent model risk mitigants, for models which have identified deficiencies.
    • Post-model adjustments (PMAs): A clearly defined, consistent, firm-wide process for applying PMAs to address model identified limitations, and for independent review of such PMAs.
    • Restrictions on model use: These should be placed when significant model deficiencies and/or errors are identified during the validation process, or if model performance tests show significant deficiencies. These could be in the form of limited model use, model controls, and mitigants.
    • Exceptions and escalations: Firms should formulate the exceptions they would allow for model use and performance and should formally implement approved policies and procedures setting out the escalation process to manage these exceptions.



Model lifecycle

 

Bridging the expectations

 

In our view, most major banks and regulated entities in the UK already have well-specified MRM practices. However, they would still need to focus on the following areas to meet the expectations of the new guidance in full:

 

  • Judiciously define the ‘proportionate implementation’ scope of the MRM function, based on the size and organisational complexity of the regulated entity, the magnitude, variety (in terms of geography, instruments etc.) and materiality of portfolio exposures to be quantified by risk models, and the number and complexity of models.
  • Improve business leadership oversight over MRM practices by specifying requirements and procedures for board oversight over the MRM function and SMF involvement in MRM.
  • Update, standardise and enhance existing MRM policies and procedures around model identification, risk rating, inventory management, effective model governance, model development and implementation, model validation, ongoing model monitoring and model risk mitigants.
  • Upgrade policy to specifically incorporate identification and mitigation of imminent risks from application of innovations such as AI/ML as well as the dynamically evolving and ever-changing environment in financial services.
  • Ensure higher efficacy, review and reporting transparency of the audit function (third line of defense), as it applies to MRM.
  • Set up an effective and recurring MRM self-assessment process along with a procedure for remediation and reporting of identified gaps.

 

In our view, a well-established, recurring process for self-assessment of MRM practices proposed by the PRA will ensure that banks are evaluating and updating their MRM frameworks, policies, and operating processes on an ongoing basis to meet the new challenges that arise from innovations and unanticipated shocks in a ceaselessly evolving risk landscape.

 

With application of AI/ML, specific attention will have to be given to widely discussed issues, which exacerbate model risk of applying such methods, such as bias, interpretability, explainability of algorithms and data adequacy and quality, which have not been covered explicitly in the SS.

 

Similarly, firms will have to identify model risks that are specific to new technological developments such as digitisation, robotics and process automation and develop well-defined policies, standards and practices around managing such risks.

Other blogs from the author

Back to CRISIL Blogs