MedTsLLM: Medical Time Series Analysis Using Multimodal LLMs

Area of Science:

• Biomedical Engineering
• Artificial Intelligence
• Clinical Informatics

Background:

• Traditional machine learning struggles with heterogeneous biomedical data.
• Unstructured clinical text is crucial but inaccessible to standard time series models.
• Integrating physiological signals with clinical context is vital for accurate patient assessment.

Purpose of the Study:

• To develop a multimodal model (MedTsLLM) for analyzing biomedical time series data.
• To bridge the gap between numerical physiological signals and unstructured clinical text using large language models (LLMs).
• To improve clinical understanding and decision-making through integrated data analysis.

Main Methods:

• Proposed MedTsLLM, a multimodal framework integrating physiological signals and clinical text via LLMs.
• Utilized patch reprogramming for time series-LLM alignment.
• Introduced novel covariate handling and contextual prompting for patient-specific information.

Main Results:

• MedTsLLM demonstrated superior performance across semantic segmentation, boundary detection, anomaly detection, and classification tasks.
• Outperformed state-of-the-art baselines on diverse datasets including ECG, respiratory monitoring, and arrhythmia detection.
• Validated the model's effectiveness in real-world clinical scenarios.

Conclusions:

• Multimodal LLMs offer transformative potential for biomedical signal analysis.
• MedTsLLM enables deeper insights from physiological data by leveraging comprehensive clinical context.
• Enhanced diagnostic accuracy, patient monitoring, and personalized treatment decisions are achievable.