What topological and geometric structure do biological foundation models learn? Evidence from 141 hypotheses

Area of Science:

• Computational Biology
• Single-cell Genomics
• Machine Learning

Background:

• Biological foundation models (e.g., scGPT, Geneformer) process single-cell gene expression data.
• Understanding the internal geometric and topological structures learned by these models is crucial.
• Distinguishing biologically meaningful structure from training artifacts is a key challenge.

Purpose of the Study:

• To investigate the geometric and topological structures within biological foundation models.
• To determine if these learned structures are biologically meaningful or artifacts of training.
• To employ autonomous hypothesis screening for large-scale investigation.

Main Methods:

• An AI-driven executor-brainstormer loop screened 141 geometric and topological hypotheses.
• Methods included persistent homology, manifold distances, cross-model alignment, and community structure analysis.
• Explicit null controls and disjoint gene-pool splits were utilized for rigorous validation.

Main Results:

• Models learn genuine geometric structure, with non-trivial topology in gene embeddings and a distance hierarchy.
• Manifold-aware metrics outperformed Euclidean distance for identifying regulatory gene pairs.
• Structure is shared across independently trained models (scGPT, Geneformer) with high alignment, but precise gene placement differs.
• Robust topological signal is concentrated in immune tissues, while lung tissue signals are fragile under stringent controls.

Conclusions:

• Biological foundation models capture genuine, shared geometric and topological structure in gene expression data.
• The precise localization of genes within this learned space is less consistent and more context-dependent.
• Autonomous screening effectively differentiates real biological structure from statistical artifacts in model representations.