Related Experiment Videos
Failure analysis of MetaMap Transfer (MMTx)
Guy Divita1, Tse Tse, Laura Roth
1Lister Hill National Center for Biomedical Communications, National Library of Medicine, Bethesda, MD 20894, USA. divita@nlm.nih.gov
Studies in Health Technology and Informatics
|September 14, 2004
Summary
The MetaMap Transfer (MMTx) tool shows promise for extracting medical terms but requires further development. Failures often stem from missing world knowledge, highlighting areas for future research in automated medical term mapping.
Area of Science:
- Medical Informatics
- Natural Language Processing
Background:
- Automated term extraction and mapping to standardized vocabularies like the Unified Medical Language System (UMLS) are crucial for organizing biomedical information.
- Existing tools like MetaMap have been evaluated, but newer iterations require performance assessment.
Purpose of the Study:
- To evaluate the performance of the MetaMap Transfer (MMTx) tool in extracting and mapping terms to UMLS concepts.
- To identify and categorize the types of mapping failures encountered by MMTx.
Main Methods:
- A pilot study involved five participants manually annotating two consumer health documents.
- Manual annotations served as a gold standard to compare against MMTx and MetaMap automated outputs.
- A failure analysis was performed on MMTx's incorrect mappings.
Main Results:
- MMTx performance was compared to MetaMap, noting differences in term extraction and mapping.
- The most common failure (30%) was attributed to missing inferential or world knowledge.
- Other failure categories requiring further research include handling of conjunctions, co-reference, and word sense.
Conclusions:
- MMTx demonstrates potential but has limitations, particularly with knowledge-intensive tasks.
- Rectifiable failures may be addressed with improved retrieval strategies.
- Complex linguistic challenges necessitate further research for enhanced automated term mapping.