Concept mask-aware pruning and augmentation for few sample model compression

Few sample model compression aims to compress models using only minimal training data. Prior studies focused on minimizing accuracy degradation under limited samples or designing specialized pruning criteria, but they largely overlooked two critical factors: pruning configuration impacts and leveraging uncompressed models for sample diversity. Inspired by deep learning interpretability, we identify the positive role of the region of interest in feature maps linked to filters. This paper refers to the binary representations of these regions as concept masks and proposes concept mask-aware (CMa) pruning and CMaMix augmentation. The former determines block-wise pruning configuration/pruning criterion by measuring concept mask divergence between input/output feature maps of each block. The latter mixes input images' salient regions using the concept mask calibrated via the uncompressed model, enhancing sample diversity to recover the performance of pruned models. The proposed method uniquely unifies compression and recovery, supporting flexible pruning granularity (filter- or block-level). Extensive experiments on image classification and semantic segmentation across diverse architectures establish new state-of-the-art results. With only 50 available samples, our method achieves classification accuracy improvements of 0.2% to 8.1% over prior works. The code is at https://github.com/sunyafeng1996/CMa-FSMC.