TabMixer: advancing tabular data analysis with an enhanced MLP-mixer approach

Tabular data, prevalent in relational databases and spreadsheets, is fundamental across fields like healthcare, engineering, and finance. Despite significant advances in tabular data learning, critical challenges remain: handling missing values, addressing class imbalance, enabling transfer learning, and facilitating feature incremental learning beyond traditional supervised paradigms. We introduce TabMixer, an innovative model that enhances the multilayer perceptron (MLP) mixer architecture to address these challenges. TabMixer incorporates a self-attention mechanism, making it versatile across various learning scenarios including supervised learning, transfer learning, and feature incremental learning. Extensive experiments on eight public datasets demonstrate TabMixer's superior performance over existing state-of-the-art methods. Notably, TabMixer achieved substantial improvements in ANOVA AUC across all scenarios: a 4% increase in supervised learning (0.840 to 0.881), 8% in transfer learning (0.803 to 0.872), and 4% in feature incremental learning (0.806 to 0.843). TabMixer demonstrates high computational efficiency and scalability through reduced floating-point operations and learnable parameters. Moreover, it exhibits strong resilience to missing values and class imbalances through both its architectural design and optional preprocessing enhancements. These results establish TabMixer as a promising model for tabular data analysis and a valuable tool for diverse applications.