MultiTabPFN: Codebook-based extensions of TabPFN for high-class-count tabular classification

Area of Science:

• Machine Learning
• Data Science
• Artificial Intelligence

Background:

• Tabular data is prevalent, and TabPFN offers fast, training-free predictions.
• TabPFN's application to high-class-count classification is limited by computational costs of fine-tuning or retraining.

Purpose of the Study:

• To extend TabPFN's capabilities to high-class-count tabular classification using a training-free approach.
• To introduce a novel framework, MultiTabPFN, addressing limitations in scaling foundation models for tabular data.

Main Methods:

• Framing multiclass prediction within the Error-Correcting Output Codes (ECOC) paradigm.
• Introducing MultiTabPFN, a modular framework featuring Classwise Principal components-based Indexing (CPI) for codebook design.
• Utilizing confidence-aware decoding for improved prediction accuracy.

Main Results:

• The novel CPI codebook method encodes class-level geometry, balancing separability and redundancy.
• MultiTabPFN consistently outperforms standard ECOC baselines on synthetic and real-world benchmarks.
• Demonstrates a practical, training-free extension of TabPFN for many-class tabular classification.

Conclusions:

• MultiTabPFN provides a principled method for scaling tabular foundation models to high-class-count settings.
• The framework offers a computationally efficient and effective solution for complex classification tasks.
• Establishes a new benchmark for training-free, high-class-count tabular classification with foundation models.