Related Experiment Videos
Developing a corpus of clinical notes manually annotated for part-of-speech
Serguei V Pakhomov1, Anni Coden, Christopher G Chute
1Division of Biomedical Informatics, Mayo College of Medicine, Rochester, MN 55905, USA. Pakhomov.Serguei@mayo.edu
International Journal of Medical Informatics
|September 20, 2005
Summary
Creating a specialized corpus of clinical text for part-of-speech (POS) tagging significantly improved accuracy. Adapting general POS taggers to medical language is essential for better performance.
Area of Science:
- Computational linguistics
- Medical informatics
- Natural Language Processing
Background:
- Developing accurate natural language processing tools for clinical text is crucial for information extraction.
- Existing part-of-speech (POS) taggers trained on general corpora may not perform optimally on specialized medical language.
Purpose of the Study:
- To construct a manually annotated corpus of clinical text for part-of-speech (POS) information.
- To train domain experts in linguistic annotation for medical notes.
Main Methods:
- Three domain experts were trained for manual annotation of clinical notes.
- A portion of the annotated corpus was combined with the Penn Treebank for training and testing POS taggers.
- Statistical POS taggers, specifically Trigrams'n'Tags (TnT), were trained and evaluated.
Main Results:
- A general English-trained TnT tagger achieved 89.79% correctness.
- Training the TnT tagger on the annotated medical corpus improved performance to 94.69%.
- Discourse-specific annotation within clinical text showed potential for further accuracy improvements.
Conclusions:
- State-of-the-art POS taggers require adaptation for the clinical text sublanguage.
- The developed annotated corpus is valuable for improving NLP tools in the medical domain.