Learning instrumental variable representation for debiasing in recommender systems

Recommender systems are essential for filtering content to match user preferences. However, traditional recommender systems often suffer from biases inherent in the data, such as popularity bias. These biases, particularly those stemming from latent confounders, can result in inaccurate recommendations and reduce both the diversity and effectiveness of the system. Existing debiasing methods for recommender systems, however, either fail to account for latent confounders or rely on predefined instrumental variables (IVs). To address this research gap, we propose a novel causality-based recommendation algorithm, Data-driven IV representation learning for debiasing in Recommender System (DIVRS), which enables the learning of IV representation directly from user-item interaction data. By leveraging the learned IV representation, DIVRS decomposes user behaviour into causal and confounding relationships to address potential bias in recommender systems. Additionally, we introduce Orthogonal Promotion Regularisation (OPR) for DIVRS to address the problem that Graph Convolutional Networks (GCNs) amplify bias. We also propose a variant of GCNs for DIVRS, called DIVRS-GCN. Experimental results on the Douban-Movie and Movielens-10M datasets demonstrate that both DIVRS and DIVRS-GCN effectively mitigate confounding bias while outperform the state-of-the-art methods in recommendation performance. For example, on both datasets, our DIVRS and DIVRS-GCN improve Recall@20 by up to 10.98 %. This validates their effectiveness and robustness. Our approaches improve recommendation accuracy while delivering more balanced and diverse suggestions, effectively addressing the limitations of existing IV-based recommender systems.