SpatialPEFT: a parameter-efficient fine-tuning framework for spatial transcriptomics foundation models

Area of Science:

• Computational Biology
• Genomics
• Bioinformatics

Background:

• Spatial transcriptomics enables high-resolution mapping of gene expression within tissues.
• Adapting large foundation models for spatial transcriptomics is computationally demanding.
• Existing methods require substantial computational resources, limiting accessibility.

Purpose of the Study:

• To develop a parameter-efficient fine-tuning framework for large spatial transcriptomics foundation models.
• To enable robust adaptation of these models on consumer-grade hardware.
• To improve downstream spatial annotation accuracy.

Main Methods:

• Introduced SpatialPEFT, a unified framework integrating Low-Rank Adaptation (LoRA), gradient checkpointing, and a spatial-aware adapter.
• Designed to reduce peak Video Random Access Memory (VRAM) requirements.
• Tested on large foundation models up to 1.4 billion parameters.

Main Results:

• Achieved over 87% reduction in peak VRAM usage.
• Enabled fine-tuning of large models on a single 16GB consumer-grade GPU.
• Demonstrated substantial improvements in downstream spatial annotation accuracy.

Conclusions:

• SpatialPEFT provides an efficient and accessible solution for adapting large spatial transcriptomics models.
• The framework significantly lowers hardware barriers for researchers in the field.
• Enhanced accuracy in spatial annotation opens new avenues for biological discovery.