K-attention: a biologically informed attention operator for data-efficient sequence-based omics modeling

Deep learning-based modeling of omics data often suffers from insufficiency and heterogeneity of the data itself. As a step towards addressing these issues, we present K-attention, a biologically informed operator that models interactions between sequence fragments effectively and efficiently. Across both biologically informed simulated datasets and two real-world omics tasks, K-attention-based networks consistently outperform canonical convolutional neural network (CNN)- and Transformer-based models, with the largest gains observed in low-data regimes. Collectively, these results indicate that K-attention enables data-efficient and biologically grounded modeling under real-world constraints.