Simple controls exceed best deep learning algorithms and reveal foundation model effectiveness for predicting genetic...

Area of Science:

• Genomics
• Computational Biology
• Pharmacology

Background:

• Modeling genetic perturbations and their impact on the transcriptome is crucial for pharmaceutical research.
• Deep learning (DL) models, particularly transformer-based foundation models, show promise for predicting these complex responses.
• A lack of clear benchmarks hinders the evaluation and understanding of these advanced models.

Purpose of the Study:

• To introduce a simple, effective baseline method for predicting post-perturbation transcriptome responses.
• To establish a necessary benchmark for comparing predictive models in perturbation studies.
• To explore the practical utility of foundation models for transcriptome-wide prediction tasks.

Main Methods:

• Development of a novel, simple baseline prediction method.
• Benchmarking against state-of-the-art deep learning and simpler neural architectures.
• Conducting generalizable fine-tuning experiments with transformer-based foundation models.
• Dataset curation and correction for perturbation prediction benchmarking.

Main Results:

• The proposed baseline method surpasses current state-of-the-art deep learning and other neural network approaches.
• Fine-tuning experiments demonstrate the generalizability and utility of foundation models for perturbation prediction.
• A corrected and improved dataset for benchmarking is provided.

Conclusions:

• The developed baseline method sets a new standard for evaluating perturbation prediction models.
• Foundation models show significant potential for transcriptome-wide prediction tasks through adaptable fine-tuning.
• This work provides essential control procedures and context for future deep learning model development in perturbation biology.