Semantic-guided Causal Representation Learning for Personalized Visual Multiple Clustering

Personalized visual multiple clustering aims to uncover multiple clustering solutions for image data based on user-specified perspectives. Existing methods typically use pre-trained Vision-Language Models to align image features with the perspective keywords for clustering. However, the pre-trained models fail to generate user-specified image representations due to the entanglement of features from different perspectives, thereby hindering the alignment between visual representations and the given keywords and ultimately degrading clustering performance. To address this, we propose Semantic-guided Causal Representation learning for Personalized Multiple Clustering (SCR-PMC), which disentangles different perspective features via semantic-guided image interventions, thereby obtaining causal visual representations for clustering. Specifically, we first construct a cross-modal structural causal model to reveal spurious correlations across different perspectives and identify the backdoor paths that affect visual representations. Following the principle of backdoor adjustment, we generate semantic-guided cross-modal confounder proxies carrying non-target perspective semantic information to construct intervention images. By enforcing representation and neighborhood consistency between the original images and their intervention images, we reduce the influence of non-target perspective factors and obtain causal visual representations. Finally, we use cross-modal alignment loss to learn optimal semantic representations that align with causal visual representations for clustering. Theoretical analysis further proves that SCR-PMC mitigates interference from non-target perspectives in target-specific representation learning. Extensive experiments demonstrate that SCR-PMC achieves competitive performance compared to existing methods on personalized visual multiple clustering tasks.