MCN: A comprehensive corpus for medical concept normalization.

Area of Science:

• Medical Informatics
• Natural Language Processing
• Clinical Data Management

Background:

• Clinical text normalization links varied expressions to standardized vocabulary terms.
• Existing annotated corpora for normalization are scarce, limited in scope (diseases/disorders), and lack wide coverage.

Purpose of the Study:

• To describe the methodology for creating a new, manually annotated, wide-coverage corpus for clinical concept normalization: the Medical Concept Normalization (MCN) corpus.
• To address limitations of previous annotation efforts by developing a novel normalization method for handling ambiguous or multi-concept named entities.

Main Methods:

• Developed a manual annotation methodology for wide-coverage clinical concept normalization.
• Applied normalization to medical problems, treatments, and tests from a prior named entity corpus.
• Implemented a novel method for splitting ambiguous named entities instead of assigning multiple labels or no label.
• Created a sieve-based normalization baseline combining MetaMap and exact match components.

Main Results:

• The Medical Concept Normalization (MCN) corpus contains 100 discharge summaries with 10,919 concept mentions normalized to 3792 unique concepts.
• Achieved inter-annotator agreement of 67.69% (pre-adjudication) and 74.20% (post-adjudication).
• The sieve-based normalization baseline achieved 77% accuracy in cross-validation.

Conclusions:

• The MCN corpus provides a valuable, wide-coverage resource for advancing clinical concept normalization research.
• The developed normalization methodology and baseline demonstrate improved handling of ambiguity in clinical text.
• The corpus will be publicly released to foster development of better normalization methods.