Related Experiment Videos
Evaluation of a belief-network-based reminder system that learns from utility feedback
1Section of Medical Informatics, University of Pittsburgh, USA.
Proceedings. Symposium on Computer Applications in Medical Care
|January 1, 1995
Summary
PRETRIEVE, a machine learning information retrieval system, demonstrated superior document ordering using utility feedback over relevance feedback. Its performance was better than random but not ideal in a medical context.
Area of Science:
- Information Science
- Medical Informatics
- Artificial Intelligence
Background:
- Information retrieval systems are crucial in medical settings.
- Machine learning enhances system performance based on user input.
- Evaluating document utility is key for effective information access.
Purpose of the Study:
- To evaluate the document ordering and retrieval performance of the PRETRIEVE system.
- To compare utility feedback with relevance feedback in an information retrieval context.
- To assess the impact of time cost modeling on retrieved document utility.
Main Methods:
- Developed a test collection with 410 expert judgments of document utility in medical order-entry.
- Measured interrater and intrarater reproducibility to validate judgments.
- Created a novel measure for document ordering quality, analogous to ROC analysis.
Main Results:
- PRETRIEVE's ordering performance was significantly better than random and superior to systems using relevance feedback.
- The system's performance was effective, though not reaching an ideal state.
- A PRETRIEVE version modeling time cost retrieved higher utility documents than a rule-based system.
Conclusions:
- Utility feedback is more effective than relevance feedback for PRETRIEVE's document ordering.
- The PRETRIEVE system shows promise for improving information retrieval in specialized domains.
- Modeling time cost enhances the utility of retrieved documents in practical applications.