Active learning algorithm for alleviating the user cold start problem of recommender systems

A key challenge in recommender systems is how to profile new users. A popular solution for this problem is to use active learning strategies. These strategies request ratings for a small set of carefully selected items to reveal the preferences of new users. In this paper, we propose a new decision tree-based algorithm for selecting these items. Treating the recommender system as a black box, the ratings collected from interviewing new users are passed on to the recommender system with the intention of improving its performance. Extensive offline evaluation with two data sets and various recommender algorithms shows that our algorithm does indeed improve the performance of the underlying recommender algorithm if users are able to rate most of the items that are presented to them during the interview. However, online evaluation with 50 real users could not prove that our algorithm does indeed have a positive impact on the performance of the underlying recommender algorithm. This reveals the discrepancy between offline and online evaluations of active learning techniques applied in the context of recommender systems. This is due to the fact that real users are not always able to rate the item selected by the active learning algorithm and therefore cannot provide the requested information, in contrast to many machine learning scenarios where the labeling of all samples is possible. Hence, further research is required to provide more certainty regarding the impact of active learning strategies on recommender algorithms.