Assessing genetic counseling efficiency with natural language processing.

Area of Science:

• Health Services Research
• Computational Linguistics
• Genetics

Background:

• Increasing referrals to genetic counseling (GC) necessitate improved efficiency.
• Characterizing GC efficiency and phase is crucial for optimizing service delivery.
• Existing methods for measuring GC efficiency are limited in scalability and real-world applicability.

Purpose of the Study:

• To develop and validate natural language processing (NLP) strategies for measuring genetic counseling (GC) efficiency.
• To classify GC measures according to the phase of genetic testing (pre- or post-).
• To apply NLP models to a large dataset of GC notes to generate real-world evidence on GC time.

Main Methods:

• Annotation of 800 GC notes from 7 clinical specialties for NLP model development.
• Extraction of GC efficiency measures (direct/indirect time, GC phase) using NLP.
• Validation of NLP models with high performance (F1 scores of 0.95 for time, 0.90 for phase).
• Application of validated models to 24,102 GC notes (2016-2023).

Main Results:

• NLP models demonstrated high accuracy in extracting GC efficiency metrics.
• Median direct time in GC was 50 minutes.
• Significant variations in GC direct time were observed across clinical specialties, time periods, and delivery modes (in-person vs. telehealth).

Conclusions:

• NLP offers a practical, scalable strategy for generating real-world evidence on GC efficiency.
• The developed NLP approach can inform research on interventions to enhance GC efficiency.
• The principles of this NLP strategy may be applicable to health services research in other domains.