Multi-timescale representation with adaptive routing for deep tabular learning under temporal shift.

Area of Science:

• Machine Learning
• Data Science
• Artificial Intelligence

Background:

• Tabular datasets in real-world applications frequently experience temporal shifts, which can significantly degrade the performance of long-range neural networks.
• Current temporal encoding and adaptation methods often treat time cues as static auxiliary variables, failing to capture the multi-horizon and heterogeneous nature of temporal dynamics.

Purpose of the Study:

• To introduce TARS (Temporal Abstraction with Routed Scales), a novel plug-and-play method designed for robust tabular learning that effectively handles temporal shifts.
• To develop a method applicable to various deep learning model backbones, enhancing their temporal robustness.

Main Methods:

• TARS employs an explicit temporal encoder to decompose timestamps into short-term, mid-term, and long-term embeddings using structured memory.
• An implicit drift encoder tracks higher-order distributional statistics to generate drift signals reflecting ongoing temporal dynamics.
• A drift-aware routing mechanism adaptively weights temporal pathways based on current conditions, integrating routed temporal representations with original features via a feature-temporal fusion layer.

Main Results:

• TARS demonstrated consistent outperformance over competitive methods across eight real-world datasets from the TabReD benchmark.
• Achieved significant average relative improvements, including +2.38% on MLP and +4.08% on DCNv2.
• Ablation studies confirmed the significant and complementary contributions of all four TARS modules.

Conclusions:

• TARS effectively improves the temporal robustness of existing deep tabular models by dynamically adapting to evolving data.
• The proposed method offers a versatile solution for enhancing the performance of various deep learning architectures in the presence of temporal shifts.