Balancing Model Complexity and Clinical Deployability in Deep Learning for Sociodemographic Information Extraction.

Area of Science:

• Natural Language Processing (NLP)
• Machine Learning
• Health Informatics

Background:

• Sociodemographic factors significantly impact health outcomes and disparities.
• Electronic medical records (EMR) often have sparse documentation of these factors in unstructured text.
• Automated extraction is challenging for clinical decision-making and health equity research.

Purpose of the Study:

• To systematically evaluate and compare six convolutional neural network (CNN) architectures for classifying sociodemographic characteristics from EMR text.
• To assess the influence of model complexity and lexical diversity on classification performance.
• To identify optimal models for efficient and interpretable clinical NLP pipelines.

Main Methods:

• Utilized data from 4375 patients across 96 primary care clinics.
• Employed six CNN architectures, including hybrid models, for binary classification tasks.
• Evaluated performance using F1 score, precision, recall, AUC-PR, and Matthews correlation coefficient, with high inter-rater reliability for manual annotation.

Main Results:

• Simpler architectures, particularly single-layer CNNs, consistently outperformed deeper or hybrid models across most characteristics (F1 score: 90.99%).
• Simpler models showed superior performance under data imbalance and varied documentation patterns.
• Hybrid models were more effective for well-documented factors but less so for sparse or diverse characteristics.

Conclusions:

• Simple CNN architectures offer a practical framework for developing efficient clinical NLP pipelines for sociodemographic data extraction.
• Findings inform model selection for real-world health equity and EMR research applications.
• The study highlights the trade-offs between model complexity and performance in diverse EMR data contexts.