Clustering Similar Diagnosis Terms.

Area of Science:

• Natural Language Processing
• Clinical Informatics
• Machine Learning

Background:

• Clinical diagnosis lists contain numerous syntactic variants, complicating automated analysis.
• Clustering these variants is crucial for improving data quality and downstream applications.

Purpose of the Study:

• To compare a string similarity heuristic with a deep learning approach for clustering syntactic variants in a clinical diagnosis list.
• To evaluate methods for improving the accuracy of clinical term normalization.

Main Methods:

• Levenshtein distance (LD) was applied to common words, excluding acronyms and numerals.
• Pair-wise substring expansions were incorporated into the string similarity heuristic.
• A deep learning model, trained on a German medical language model, was also evaluated.

Main Results:

• The enhanced string similarity heuristic (LD on common words + substring expansions) improved F1 score by 13% above the baseline (plain LD), reaching a maximum F1 of 0.71.
• The deep learning-based approach did not surpass the baseline, achieving a maximum F1 score of 0.42.

Conclusions:

• A carefully designed string similarity heuristic can outperform deep learning models for specific clinical text processing tasks.
• The proposed heuristic offers a more effective method for clustering syntactic variants in clinical diagnosis lists.