Related Experiment Videos
A randomized double-blind controlled trial of automated term dissection
P L Elkin1, K R Bailey, P V Ogren
1Mayo Foundation, Rochester, MN, USA.
Proceedings. AMIA Symposium
|November 24, 1999
Summary
Automated Term Dissection (ATD) and Human Term Dissection (HTD) show similar accuracy in representing semantic dependencies. ATD may be preferable for complex terms, offering a useful alternative for accurate terminological expressions.
Area of Science:
- Medical Informatics
- Natural Language Processing
- Clinical Terminology
Background:
- Accurate representation of semantic dependencies in compositional expressions is crucial for clinical terminology.
- Existing methods for term dissection can be labor-intensive and subject to inter-observer variability.
Purpose of the Study:
- To compare the accuracy of an automated mechanism for term dissection (ATD) against a practicing Internist (HTD).
- To evaluate inter-observer variability and failure analysis consistency between ATD and HTD methods.
Main Methods:
- 500 terms requiring compositional expressions were randomly divided into two sets.
- Set A was processed using the Automated Term Dissection (ATD) Algorithm.
- Set B was dissected by a physician using Human Term Dissection (HTD).
- Expert indexers reviewed dissected terms, assessing accuracy and performing failure analysis.
Main Results:
- ATD accuracy: 62.7% (265/424); HTD accuracy: 65.7% (272/414).
- No statistically significant difference in acceptability between ATD and HTD (p = 0.33).
- A trend towards higher ATD acceptability for complex terms (≥3 elements): 53.6% vs. 43.6% (p = 0.11).
- Both methods misrepresented kernel concepts and modifiers more than qualifiers (p < 0.001).
Conclusions:
- No significant difference in accuracy and readability between ATD and HTD.
- ATD shows a non-significant trend towards improved performance in complex terms.
- Automated term dissection is a viable and potentially preferable method for accurate terminological expressions.