Edge-updating graph neural networks for modeling feature interactions in tabular data.

Area of Science:

• Machine Learning
• Graph Neural Networks
• Data Science

Background:

• Tabular data analysis is crucial in machine learning.
• Existing Graph Neural Networks (GNNs) face challenges like oversmoothing when applied to tabular data.
• Gradient-boosted decision trees (e.g., XGBoost, CatBoost) are state-of-the-art for tabular data.

Purpose of the Study:

• To propose a message-passing GNN based on Graph Isomorphism Network (GIN) for enhanced tabular data learning.
• To model complex feature interactions within tabular datasets.
• To address the oversmoothing problem common in GNNs.

Main Methods:

• Constructed fully-connected, unweighted feature graphs from tabular data using contextual feature encodings.
• Incorporated a classification node for graph representation during inference.
• Employed a neural network for edge attribute learning and utilized residual connections for node and edge updates to mitigate oversmoothing.

Main Results:

• Achieved the best mean rank among 6 tabular deep learning and GNN models across 12 datasets.
• Outperformed XGBoost and CatBoost on all datasets with default hyperparameters and on 8 datasets with tuned hyperparameters.
• Outperformed 5 commonly used or recently-proposed GNNs on all tested datasets.

Conclusions:

• The proposed GNN architecture demonstrates superior performance on tabular data compared to existing deep learning and GNN models.
• The model effectively handles feature interactions and mitigates oversmoothing, offering a competitive alternative to gradient-boosted trees.
• This GNN approach provides a powerful new tool for tabular data machine learning tasks.