Quantized multi-task learning for context-specific representations of gene network dynamics.

Area of Science:

• Computational Biology and Bioinformatics
• Genomics and Transcriptomics
• Machine Learning in Biology

Background:

• Gene networks are dynamic and context-dependent, not static entities.
• Existing models often fail to capture the nuanced, context-specific nature of gene regulation.
• Understanding gene network dynamics is crucial for disease modeling and therapeutic target identification.

Purpose of the Study:

• To develop a multi-task learning strategy for generating context-specific representations of gene network dynamics.
• To create a foundational model from non-malignant single-cell transcriptomes and fine-tune it for cancer-specific analysis.
• To identify candidate therapeutic targets within the colorectal tumor microenvironment.

Main Methods:

• Assembled a large corpus of ~103 million human single-cell transcriptomes.
• Employed a two-stage pretraining strategy: foundational model on non-malignant cells, followed by continual learning on cancer cells.
• Utilized multi-task learning for context-specific representations across various cell types, tissues, and diseases, including model quantization for efficiency.

Main Results:

• Successfully generated context-specific gene network representations.
• The cancer-tuned model effectively learned cell states and predicted tumor-restricting factors in the colorectal tumor microenvironment.
• Model quantization enabled efficient fine-tuning and inference while retaining biological insights.

Conclusions:

• Multi-task learning provides a powerful framework for context-specific disease modeling.
• This approach can yield contextual predictions of candidate therapeutic targets for human diseases.
• The developed strategy enhances our ability to understand and target complex biological systems.