SIGMA: self-supervised inference of gene networks via masked auto-encoding

Inferring gene regulatory networks (GRNs) from expression profiles is essential for identifying critical genes within complex disease pathways. However, current machine learning-based GRN inference methods face two challenges. Unsupervised methods struggle to achieve satisfactory accuracy in inference, while supervised methods are limited by the scarcity of high-quality interaction labels. Further, existing models demonstrate significant shortcomings when it comes to transferring reasoning to other GRN task subtypes. These issues affect GRN inference and hinder the ability to discover new regulatory patterns.

To address these challenges, we have developed SIGMA: a transformer-based framework that uses self-supervised learning to pretrain the encoder on expression profiles. This alleviates the need for high-quality labels. During pretraining, it converts gene expression pairs into non-overlapping patches, and randomly masks some of these patches. This forces the encoder to extract correlation representations from the unmasked patches without label guidance, enabling the decoder to reconstruct the masked patches while preserving their similarity. Experiments have demonstrated that the pretrained encoder can accurately infer GRNs and be used to infer other subtypes, thereby reducing reliance on labels. Benchmark tests on human and mouse datasets have shown that SIGMA outperforms state-of-the-art methods. When applied to breast cancer datasets, SIGMA produced predictions that were consistent with established networks and identified candidate interactions that were not present in the gold-standard networks. Further investigation and experimental validation of these relationships is warranted.