Representation Learning for Tabular Data: A Comprehensive Survey

Area of Science:

• Machine Learning
• Data Science
• Artificial Intelligence

Background:

• Tabular data is prevalent in machine learning for classification and regression.
• Deep Neural Networks (DNNs) show promise in tabular data through representation learning.

Purpose of the Study:

• To systematically survey the field of tabular representation learning.
• To cover background, challenges, benchmarks, and DNNs' pros and cons.
• To organize existing methods based on generalization capabilities.

Main Methods:

• Categorization of models into specialized, transferable, and general.
• Hierarchical taxonomy for specialized models (features, samples, objectives).
• Exploration of strategies for feature and sample representations.

Main Results:

• Detailed strategies for high-quality representations in specialized models.
• Pre-training and fine-tuning approaches for transferable models.
• Adaptation strategies for general tabular foundation models across datasets.

Conclusions:

• Discussion of ensemble methods and extensions like open-environment learning and multimodal learning.
• Overview of tabular understanding tasks.
• Provides a comprehensive resource for tabular representation learning research.