SEMI-CAVA: A Causal Variational Approach to Semi-Supervised Learning

Deep learning has advanced rapidly, but relies heavily on large-labeled datasets for effective training. This is particularly challenging in fields like medicine, where expert labeling is costly, labor-intensive, and prone to bias and error. Semi-supervised learning (SSL) addresses this challenge by reducing reliance on labeled data. SSL is closely tied to the concept of causation. However, recent works relating causality to SSL are limited by modeling only low-dimensional observations or designing a plug-in module to alleviate the class imbalance. In this paper, we take steps towards training causal generative models for semi-supervised learning, combining principles from causality and variational inference. We interpret the Mixup strategy as a stochastic intervention and introduce a consistency loss to promote coherent latent representations. Under reasonable assumptions, we provide theoretical guarantees that the learned latent representations align with true causal factors up to permissible ambiguities. The experimental results show the proposed approach achieves state-of-the-art performance on several medical datasets of different modalities. Additionally, we test our model on standard benchmarking datasets: CIFAR10, CIFAR100, and SVHN, where it achieves competitive performance.