Bayesian Hyperparameter Optimization Improves scGPT Fine-Tuning for Single-Cell Multi-Omics Integration.

Area of Science:

• Computational Biology
• Single-cell Multi-omics Analysis
• Foundation Models

Background:

• Foundation models like scGPT show promise for single-cell multi-omics integration but are sensitive to hyperparameter settings.
• Manual fine-tuning is costly, dataset-specific, and often irreproducible.
• Robust hyperparameter optimization strategies for these models are underdeveloped.

Purpose of the Study:

• To develop and evaluate a Bayesian optimization framework for automated scGPT fine-tuning.
• To improve the stability, reproducibility, and performance of scGPT in multi-omics integration tasks.

Main Methods:

• Developed a Bayesian optimization framework using Tree-structured Parzen Estimators (TPE).
• Applied the framework to fine-tune scGPT on two benchmark bone marrow mononuclear cell (BMMC) multi-omics datasets (CITE-seq and GSE194122).
• Evaluated performance using biological conservation and batch integration metrics.

Main Results:

• Bayesian optimization consistently improved biological conservation (AvgBIO) and batch integration (PCR, ARI) compared to default scGPT settings.
• Significant performance gains observed on the GSE194122 dataset, improving AvgBIO from 0.19 to 0.60 and ARI from 0.007 to 0.63, while reducing validation loss.
• Demonstrated dataset-specific sensitivity of scGPT fine-tuning, highlighting the need for automated optimization.

Conclusions:

• Bayesian optimization offers an effective and reproducible strategy for stabilizing scGPT fine-tuning in single-cell multi-omics.
• Emphasizes the critical role of systematic optimization for enhancing the robustness of foundation models in computational biology.
• The developed model and dataset are publicly available for broader application.