A hybrid rule-based NLP and machine learning approach for PII detection and anonymization in financial documents

Area of Science:

• Computer Science
• Data Security
• Natural Language Processing

Background:

• Financial documents contain sensitive Personally Identifiable Information (PII) requiring robust protection.
• Data breaches and non-compliance pose significant risks to financial institutions.
• Existing PII anonymization methods may lack scalability and accuracy.

Purpose of the Study:

• To develop and evaluate a scalable hybrid approach for detecting and anonymizing PII in financial documents.
• To enhance the security and regulatory compliance of financial data.
• To improve upon current methods for PII protection in operational financial contexts.

Main Methods:

• Integration of rule-based Natural Language Processing (NLP), Machine Learning (ML), and a custom Named Entity Recognition (NER) model.
• Creation of a diverse synthetic dataset mimicking real financial documents for training and validation.
• Utilizing confusion matrices, ROC curves, and precision-recall curves for comprehensive model evaluation.

Main Results:

• The hybrid model achieved 94.7% precision, 89.4% recall, and 91.1% F1-score on synthetic data.
• An overall accuracy of 89.4% was recorded on synthetic datasets, and 93% on real financial documents.
• The model demonstrated strong generalization capabilities validated through rigorous performance metrics.

Conclusions:

• The proposed hybrid approach offers a robust and efficient solution for PII anonymization in financial documents.
• This method significantly enhances the protection of sensitive information in operational financial settings.
• The findings support the adoption of advanced NLP and ML techniques for improved data security and compliance.