Quadruplet Augmentation With Attribute and Structure Invariance for Online Continual Learning

Online Continual Learning (OCL) learns from non-independently and identically distributed streaming data with unknown task boundaries during training and testing. Previous methods suffer from the shortcut feature trap and limited plasticity, leading to two requirements: attribute invariance and structure invariance. The former requires to capture the attributes of objects which maintain invariance during all sessions of OCL, while the latter requires to capture the relation of different attributes during OCL. From the causal invariant representation perspective, we propose Quadruplet Augmentation (QuadAug) by preserving attribute and structure invariance via data and channel augmentation with four types of augmentation strategies. First, we build a fine-grained causal graph of OCL to isolate the session-invariant attributes from confounders. Then, by observing different roles of amplitude and phase components of Fourier domain during knowledge transfer, QuadAug preserves attribute invariance by an Amplitude-Phase augmentation (AP-aug) module via a bidirectional data augmentation strategy, to intervene subtle confounders: the single-session class factor and the class-irrelevant factor. Finally, by decomposing the structure invariance into two necessary conditions: channel independence and channel sufficiency, QuadAug preserves structure invariance by an Independence-Sufficiency augmentation (IS-aug) module, which preserves the channel independence property with an inter-channel discrepancy constraint, and the channel sufficiency property with an adversarial augmentation constraint. QuadAug produces significant improvement on four sequential datasets and three blurry datasets for OCL.