Bayesian hyperparameter optimization improves scGPT fine-tuning for single-cell multi-omics integration

Foundation models such as scGPT have demonstrated strong potential for single-cell multi-omics integration; however, their downstream performance is highly sensitive to hyperparameter selection. Manual fine-tuning remains computationally expensive, dataset-dependent, and often irreproducible. Despite the increasing adoption of foundation models in single-cell analysis, systematic strategies for robust hyperparameter optimization remain underexplored.

We developed a Bayesian optimization framework based on Tree-structured Parzen Estimators (TPE) for automated fine-tuning of scGPT and evaluated its performance on two benchmark bone marrow mononuclear cell (BMMC) multi-omics datasets, including CITE-seq and GSE194122 datasets. Across datasets, Bayesian optimization consistently improved biological conservation and batch integration metrics compared with default scGPT configurations. On the original BMMC benchmark, optimization improved AvgBIO from 0.59 to 0.67 and PCR from 0.33 to 0.52. On the GSE194122 dataset, the default configuration exhibited unstable convergence and weak biological preservation (AvgBIO = 0.19; ARI = 0.007), whereas Bayesian optimization substantially improved integration performance (AvgBIO = 0.60; ARI = 0.63) while reducing validation loss from 137 to 47.1. These findings demonstrate substantial dataset-specific sensitivity of scGPT fine-tuning and highlight the importance of automated optimization for stable deployment across heterogeneous multi-omics datasets. Our study demonstrates that Bayesian optimization provides an effective and reproducible strategy for stabilizing scGPT fine-tuning across diverse single-cell multi-omics datasets. Rather than introducing a new integration architecture, this work emphasizes the importance of systematic optimization for improving robustness and reproducibility of foundation-model applications in computational biology.

Our model and dataset are freely available at: https://github.com/daren642/scGPT_multiomic_tuning.