Related Experiment Video
Updated: Jul 2, 2026

A Multimodal Imaging Framework to Advance Phenotyping of Living Label-free Breast Cancer Cells
Published on: August 22, 2025
A multimodal EHR-based phenotyping framework integrating consensus clustering and transformer-based clinical NLP:
Daniele Pala1, Marco Vincenzo Lenti2, Giovanni Santacroce2
1Department of Management, Information and Production Engineering, University of Bergamo, Bergamo, Italy; Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Pavia, Italy.
Objective:
To develop and evaluate a multimodal electronic health record (EHR)-based phenotyping pipeline integrating structured and unstructured clinical data to identify disease subgroups and characterize longitudinal trajectories in a real-world setting.
Materials And Methods:
We conducted a retrospective multicenter study including 1,598 patients with autoimmune gastritis. Structured demographic and clinical variables were combined with longitudinal endoscopic and histological data extracted from routine care. A consensus clustering strategy integrating partitioning (K-medoids) and hierarchical approaches was applied to identify robust patient subgroups. Free-text endoscopic reports were processed using a fine-tuned transformer-based natural language processing (NLP) model to automatically extract structured phenotypic features. To address irregular follow-up intervals, time-normalized progression indices were developed to capture both severity and temporal dynamics of disease evolution.
Results:
After preprocessing, 607 patients were included in the analysis. The consensus clustering approach identified three clinically distinct subgroups. The NLP model demonstrated high performance in extracting endoscopic features (accuracy 90.2%, balanced accuracy 89.3%). Application of the proposed progression indices revealed significant differences in longitudinal patterns of mucosal damage across clusters (p < 0.01).
Conclusion:
This study demonstrates the feasibility of integrating clustering techniques and transformer-based clinical NLP within a unified EHR phenotyping pipeline. The proposed approach supports scalable secondary use of structured and narrative clinical data for subgroup discovery and trajectory modeling in chronic diseases.