Evaluation of Pseudonymization of Japanese Progress Notes by LLM

Area of Science:

• Natural Language Processing
• Medical Informatics
• Computational Linguistics

Background:

• Pseudonymizing clinical text is crucial for patient privacy.
• Japanese language presents unique challenges for text processing due to its agglutinative nature.
• Existing methods may not adequately address the complexities of Japanese clinical data.

Purpose of the Study:

• To develop and evaluate a prototype tool for pseudonymizing Japanese clinical text.
• To apply a large language model-based named entity recognition (NER) approach for this task.
• To assess the preliminary performance of the developed system.

Main Methods:

• Utilized the Presidio framework and the GiNZA BERT model for NER.
• Trained the model on Japanese clinical text data.
• Evaluated the system's precision, recall, and F-score.

Main Results:

• The prototype achieved a precision of 0.672, a recall of 0.995, and an F-score of 0.802.
• Demonstrated the feasibility of using LLM-based NER for Japanese clinical text pseudonymization.
• Identified areas for performance improvement.

Conclusions:

• The developed tool shows potential for effective pseudonymization of Japanese clinical text.
• Further accuracy improvements are possible with domain-specific LLM training and morphological analysis.
• This approach can contribute to enhanced data privacy in Japanese medical informatics.