Related Experiment Video
Updated: Jun 14, 2026
A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
Published on: September 20, 2018
Automated detection of follow-up appointments using text mining of discharge records
Kari L Ruud1, Matthew G Johnson, Juliette T Liesinger
1Division of Healthcare Policy and Research, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA. ruud.kari@mayo.edu
Insights
Text mining accurately identifies follow-up appointment details in hospital discharge records. This automated method enhances efficiency for performance assessment and quality research.
Area of Science:
- Health Informatics
- Clinical Data Analysis
- Natural Language Processing
Background:
- Hospital discharge summaries contain crucial follow-up appointment information.
- Manual review of these unstructured records is time-consuming and resource-intensive.
- Accurate identification of follow-up arrangements is vital for patient care continuity and quality assessment.
Purpose of the Study:
- To evaluate the accuracy of text mining in detecting specific follow-up appointment criteria within free-text hospital discharge records.
- To compare text mining performance against manual abstraction (gold standard).
Main Methods:
- A cross-sectional study was conducted at Mayo Clinic Rochester hospitals.
- Discharge records from general medicine inpatients (n=6481) were analyzed.
- SAS Text Miner software was used to identify appointment date, time, physician, or location, and results were compared to manual review.
Main Results:
- Text mining achieved 96.6% agreement with manual review for follow-up appointment arrangements.
- Overall accuracy metrics included sensitivity (96.8%), specificity (96.3%), positive predictive value (97.0%), and negative predictive value (96.1%).
- Individual criteria accuracy ranged from 82.9% (location) to 97.5% (physician).
Conclusions:
- Text mining is a highly accurate method for extracting follow-up appointment information from unstructured discharge summaries.
- This technology offers significant resource savings for performance assessment and quality improvement initiatives.
- Automated analysis of clinical notes can streamline data extraction for research and operational needs.
Objective:
To determine whether text mining can accurately detect specific follow-up appointment criteria in free-text hospital discharge records.
Design:
Cross-sectional study.
Setting:
Mayo Clinic Rochester hospitals.
Participants:
Inpatients discharged from general medicine services in 2006 (n = 6481).
Interventions:
Textual hospital dismissal summaries were manually reviewed to determine whether the records contained specific follow-up appointment arrangement elements: date, time and either physician or location for an appointment. The data set was evaluated for the same criteria using SAS Text Miner software. The two assessments were compared to determine the accuracy of text mining for detecting records containing follow-up appointment arrangements.
Main Outcome Measures:
Agreement of text-mined appointment findings with gold standard (manual abstraction) including sensitivity, specificity, positive predictive and negative predictive values (PPV and NPV).
Results:
About 55.2% (3576) of discharge records contained all criteria for follow-up appointment arrangements according to the manual review, 3.2% (113) of which were missed through text mining. Text mining incorrectly identified 3.7% (107) follow-up appointments that were not considered valid through manual review. Therefore, the text mining analysis concurred with the manual review in 96.6% of the appointment findings. Overall sensitivity and specificity were 96.8 and 96.3%, respectively; and PPV and NPV were 97.0 and 96.1%, respectively.
Analysis:
of individual appointment criteria resulted in accuracy rates of 93.5% for date, 97.4% for time, 97.5% for physician and 82.9% for location.
Conclusion:
Text mining of unstructured hospital dismissal summaries can accurately detect documentation of follow-up appointment arrangement elements, thus saving considerable resources for performance assessment and quality-related research.
Related Concept Videos
01:31Discharge Summary Forms
Here's a detailed look at the key components and guidelines for preparing a discharge summary:
01:29Issues And Trends In Healthcare Delivery System
Cost Containment
Payment for healthcare services has historically promoted adoption of costly and often unnecessary or inefficient...
01:19Purpose of Health Records II
01:30Pulse rhythm
Conversely, an irregular pulse pattern is termed dysrhythmia, stemming from disruptions in cardiac muscle...
01:17Flow Sheet
Here's a closer look at the examples of flowsheets commonly used by nurses:
Graphic Sheet Documentation:
01:25Assessment of apical radial pulse
The A-R pulse assessment involves simultaneous evaluation of the apical and radial pulses. When the apical and radial pulse rates vary, this assessment helps identify a pulse deficit.
Pre-Procedural Preparation