• Standardized list and mapped entities were not uniform as they had additional words and casing issues.
• Many entity mappings consisted of variations greater than 2. Thus, causing ambiguity.
• There were many overlapping entities, abbreviations, spelling mistakes.
• Large amount of data to be processed including those entries as well which were not in scope, causing additional noises.
• Current Rule based and similarity models failed to provide more than 20% of matching for every fresh run.
CRISIL proposed a comprehensive framework & workflow design which includes two approaches that helped incorporate business rules through data & model:
• Deep Learning approach along with sufficient historical data to capture entity name change, map with high accuracy for overlapping classes.
• Hierarchical Neural Networks (HNN) helped to establish hierarchical relationships (parent-subsidiaries).
An approach that helped to incorporate both business rules and hierarchical relationships to arrive accurate mapping for each run.