Attentive interpretable tabular learning (TabNet) for miRNA expression classification in cervical cancer and Alzheimer's disease

The classification of RNA sequencing (RNA-seq) data presents a fundamental challenge in computational biomedicine: raw read counts are high-dimensional, over-dispersed, and heteroscedastic, requiring careful preprocessing before machine learning models can be applied. Existing deep learning approaches for RNA-seq classification have largely overlooked the tabular nature of genomic expression data and have offered limited embedded interpretability. This study proposes TabNet, an attentive interpretable tabular learning architecture, as a framework for miRNA-based disease classification and evaluates its performance and interpretability against three established baseline models.

Two publicly available miRNA expression datasets were examined: a cervical cancer cohort (58 samples, 714 miRNAs) and an Alzheimer's disease cohort (70 samples, 503 miRNAs). Raw count data were preprocessed following the MLSeq framework, encompassing DESeq median-ratio normalisation, near-zero variance filtering, supervised gene ranking based on DESeq differential expression analysis, and the DESeq variance-stabilising transformation (vst). Gene subsets corresponding to 5%, 10%, and 30% of the post-filtering feature space were evaluated. TabNet was benchmarked against eight baselines spanning four learning paradigms: a 1D Convolutional Neural Network (CNN), a Long Short-Term Memory network (LSTM), Extreme Gradient Boosting (XGBoost), Random Forest, ElasticNet-regularised logistic regression, a Multi-Layer Perceptron (MLP), the prior-fitted transformer TabPFN, and the inherently interpretable Explainable Boosting Machine (EBM). Each configuration was assessed over 50 independent random train/test splits (70%/30%) under a strict leakage-free protocol in which all preprocessing, hyperparameter selection, and early stopping relied exclusively on inner validation subsets of the training partitions; all metrics are reported as mean $\pm$ SD.

Under the leakage-free protocol, a group of four methods reached a statistically indistinguishable accuracy ceiling: on the cervical cancer dataset, ElasticNet (95.06% $\pm$ 4.66), EBM (94.94% $\pm$ 4.60), TabPFN (94.71% $\pm$ 4.64), and Random Forest (94.35% $\pm$ 5.82); on the Alzheimer's disease dataset, TabPFN (70.86% $\pm$ 10.26), EBM (70.67% $\pm$ 12.67), and ElasticNet (70.48% $\pm$ 9.76) led. TabNet reached 85.18% $\pm$ 11.10 (cervical) and 64.38% $\pm$ 11.23 (Alzheimer's) at its respective best levels -- below the top tier, but with the distinctive property of producing sparse, instance-level attention explanations as part of its forward computation. Supplementary analyses showed that self-supervised pre-training conferred no significant benefit at these cohort sizes, that results were robust to the choice between supervised differential-expression and unsupervised variance feature ranking, and that instance-level attention patterns varied substantially across splits, supporting their use as illustrative rather than confirmatory evidence.

In rigorously leakage-controlled small-cohort miRNA classification, strongly regularised classical models, glass-box additive models, and prior-fitted transformers converge to the same accuracy ceiling, and no evaluated method -- including attention-based deep learning -- surpasses it. Within this landscape, TabNet remains distinctive as the only evaluated deep architecture whose sequential attention mechanism produces sparse instance-level miRNA attributions as an inherent part of prediction, offering a trade-off between embedded interpretability and a moderate accuracy cost that we quantify precisely. These results argue for protocol-first evaluation and calibrated claims when deep tabular models are applied to small transcriptomic cohorts.