Dual-Domain Division Multiplexer for General Continual Learning: A Pseudo Causal Intervention Strategy

Area of Science:

• Machine Learning
• Artificial Intelligence
• Causal Inference

Background:

• General Continual Learning (GCL) faces challenges with inter-task and intra-task biases due to non-stationary data streams.
• Existing GCL methods struggle to simultaneously address these biases, often falling into spurious correlation traps.
• Spurious correlations in GCL can exist between confounders and inputs, as well as among multiple causal variables.

Purpose of the Study:

• To propose a novel approach for mitigating biases in General Continual Learning (GCL).
• To introduce a plug-and-play module that enhances model performance and reduces catastrophic forgetting in GCL.
• To leverage causal inference and frequency transformation techniques for improved continual learning.

Main Methods:

• Formalized a structural causality model for GCL to understand spurious correlations.
• Developed the Dual-Domain Division Multiplex (D3M) unit, a plug-and-play module for GCL.
• D3M employs a two-stage pseudo causal intervention strategy using frequency and spatial domain multiplexing (FDM and SDM modules).

Main Results:

• The D3M unit effectively intervenes in confounders and multiple causal factors.
• Experiments on four datasets showed D3M significantly enhances accuracy in GCL tasks.
• D3M demonstrated a substantial reduction in catastrophic forgetting compared to existing GCL methods.

Conclusions:

• The proposed D3M unit offers a lightweight and model-agnostic solution for improving GCL.
• D3M successfully addresses both inter-task and intra-task biases by tackling spurious correlations.
• This approach advances continual learning by integrating causal inference and dual-domain feature manipulation.