Validation of "Neural Networks based Wire Payment Fraud Detection Model" for strategic and regulatory submission for a large US based Financial Institution

Client : Large US Financial Institution

 

Objective

 

To validate a neural-network-based wire payment fraud detection model for regulatory submission and enhanced strategic decision-making. The goal was to validate the bank’s model for two large US portfolios and provide a decision on the model's use.

 

CRISIL's Solution

 

CRISIL GR&RS performed validation of the wire payment fraud detection model. This involved:

  • Reviewing all documents submitted for validation and assessing whether more information was required from stakeholders to complete the thorough validation process;
  • Holding regular discussions with stakeholders to ensure all ambiguities in their reports were removed before performing additional tests - at CRISIL’s suggestion - to evaluate the model's performance.

Validation Methodology

  • Thorough study of the model development document and other documents and research articles relevant to understanding of the model;
  • Review all tests done by the developers for evaluating the neural networks model performance;
  • Prescribe and review results of performance tests other than those done by the model developer.

Validation Highlights

  • The neural network model was found to be not performing well

 

Client Impact

 

  • Model validation completed in short timeframe 
  • Significant shortcomings in model performance identified
  • CRISIL provided insights and suggestions, including new performance metrics to evaluate fraud model and enhancements to the quality of model documentation
  • Created a thorough validation document to be used by the client for CCAR submission
  • Provided specific recommendations about future use of the model in the two target portfolios

Questions

 

Looking for high-end research and risk services? Reach out to us at:

 

United States
1-855-595-2100/
+1 646 292 3520

 

United Kingdom
+44 (0) 870 333 6336

India
+91 22 33 42 3000 /
+91 22 61 72 3000