Natural language processing for scalable feature engineering and ultra-high-dimensional confounding adjustment in...

Area of Science:

• Health Informatics
• Biostatistics
• Computational Epidemiology

Background:

• Healthcare database studies often struggle with confounding due to unmeasured factors.
• Current methods for high-dimensional proxy adjustment do not utilize free-text electronic health record (EHR) notes.
• Natural Language Processing (NLP) can extract structured features from unstructured EHR data.

Purpose of the Study:

• To evaluate the impact of incorporating NLP-generated features into high-dimensional proxy adjustment.
• To assess if supplementing claims data with EHR-derived NLP features enhances confounding control.

Main Methods:

• Linked Medicare claims with EHR data to form three medication-comparison cohorts.
• Applied NLP techniques to generate features from free-text EHR notes.
• Utilized LASSO regression for propensity score model fitting with claims-only and claims+NLP feature sets.

Main Results:

• Including NLP features alongside claims data improved overall covariate balance (standardized differences < 0.1).
• Adjustment for NLP features shifted effect estimates towards expected directions in two of three studies.
• The impact on estimated treatment effects varied, with no change in one study.

Conclusions:

• Supplementing administrative claims with NLP features enhances ultra-high-dimensional proxy confounder adjustment.
• This approach offers a modest improvement in capturing relevant confounder information.
• NLP integration shows promise for strengthening causal inference in healthcare database research.