Related Experiment Videos
Evaluation of the DEFINDER system for fully automatic glossary construction
1Center for Research on Information Access, Columbia University, New York, NY 10027, USA.
Proceedings. AMIA Symposium
|February 5, 2002
Summary
DEFINDER, a rule-based system, effectively extracts definitions from articles, achieving 87% precision and 75% recall. Its output enhances dictionaries and clarifies technical terms for general users.
Area of Science:
- Natural Language Processing
- Information Extraction
- Computational Linguistics
Background:
- Existing resources for definitions are often incomplete.
- Consumer-oriented articles contain valuable, yet unmined, definitional content.
- DEFINDER addresses the need for automated definition extraction.
Purpose of the Study:
- To quantitatively and qualitatively evaluate DEFINDER, a rule-based system for extracting definitions.
- To assess DEFINDER's performance against human benchmarks and existing resources like UMLS.
- To evaluate the usability and readability of extracted definitions.
Main Methods:
- DEFINDER employs rule-based mining of full-text consumer articles.
- Quantitative evaluation used precision and recall metrics against human performance.
- Qualitative evaluation assessed user-centered criteria like usability and readability.
Main Results:
- DEFINDER achieved 87% precision and 75% recall, surpassing existing resources.
- Extracted definitions were rated higher in usability and readability than specialized dictionary definitions.
- The system demonstrated effectiveness in identifying and extracting definitions.
Conclusions:
- DEFINDER effectively extracts definitions from consumer-oriented articles.
- The system's output can enhance existing dictionaries and aid in clarifying technical terms.
- DEFINDER shows significant potential for improving access to understandable information.