Large Models for Small Tables: Adapting Tabular Foundation Models to EHR Data

Area of Science:

• Artificial Intelligence in Medicine
• Machine Learning for Healthcare
• Clinical Data Analytics

Background:

• Electronic Health Records (EHR) possess rich structured tabular data, but developing accurate predictive models for small clinical datasets is difficult.
• Foundation models, successful in natural language and imaging, are being explored for generalist medical AI but are under-explored for structured EHR data.

Purpose of the Study:

• To investigate the adaptation of a tabular foundation model for predictive tasks using EHR data.
• To evaluate the efficacy of fine-tuning a pre-trained tabular foundation model on clinical datasets.

Main Methods:

• Fine-tuning TabPFN, a Transformer-based foundation model pre-trained on synthetic tabular data.
• Applying the fine-tuned model to various tabular clinical prediction tasks.
• Comparing the performance of the foundation model against conventional methods.

Main Results:

• The fine-tuned foundation model consistently outperformed conventional methods across multiple tabular clinical prediction tasks.
• The model demonstrated higher performance, indicating its ability to capture complex patterns in clinical data.
• The study validates the 'large models for small tables' approach for EHR data.

Conclusions:

• Adapting foundation models to EHR data offers a promising avenue for advancing healthcare AI.
• Fine-tuning pre-trained tabular models like TabPFN can significantly improve predictive accuracy on limited clinical datasets.
• This approach effectively combines prior knowledge from large-scale pre-training with domain-specific EHR data.