K-attention: a biologically informed attention operator for data-efficient sequence-based omics modeling.

Area of Science:

• Computational biology
• Bioinformatics
• Machine learning for omics

Background:

• Deep learning models for omics data face challenges with data insufficiency and heterogeneity.
• Existing models like CNNs and Transformers struggle with complex biological sequence interactions.

Purpose of the Study:

• To introduce K-attention, a novel biologically informed operator for effective and efficient modeling of interactions between biological sequence fragments.
• To evaluate the performance of K-attention-based networks against established deep learning models in omics data analysis.

Main Methods:

• Development of the K-attention operator, designed to capture relationships within biological sequences.
• Comparative analysis using biologically informed simulated datasets and two real-world omics tasks.
• Benchmarking K-attention against Convolutional Neural Networks (CNNs) and Transformer-based models.

Main Results:

• K-attention-based networks demonstrated superior performance compared to CNN and Transformer models.
• Significant performance gains were observed with K-attention, particularly in low-data scenarios.
• The method proved effective in modeling interactions between sequence fragments.

Conclusions:

• K-attention offers a data-efficient and biologically grounded approach for deep learning in omics.
• The operator addresses key limitations of existing methods in handling insufficient and heterogeneous omics data.
• K-attention represents a promising advancement for biological sequence modeling under real-world constraints.