Searching to Modulate for Cold-Start Recommendation

Making personalized recommendation for cold-start users, who only have a few interaction histories, is a challenging problem in recommendation systems. Recent works leverage hypernetworks to directly map interaction histories to user-specific parameters, which are then used to modulate predictor by certain modulation structure. These works obtain the state-of-the-art performance. However, there lacks a general approach to design the modulation structure. Instead of using a fixed modulation function and deciding modulation position by expertise, we propose to determine proper modulation structure, including function and position, via neural architecture search. We propose two approaches. We first design a symbolic search space which covers broad models and theoretically prove that this search space can be transformed to a much smaller space, enabling an efficient and robust one-shot search algorithm, called ColdNAS. Since recommendation systems are a special case of bipartite matching problems, the proposed methods can be generalized to a wide range of cold-start tasks, such as disease-gene association prediction for emerging diseases. However, diverse scenarios introduce new challenges in both the flexibility of the search algorithm and the search space. To address these limitations, we further propose ColdNAS$_+$+, where we employ neural networks to model modulation functions to extend search space and design a two-stage decoupled stochastic search algorithm to enable non-differentiable targets in continuous spaces. Extensive experimental results on benchmark datasets show that modulation structures obtained by ColdNAS and ColdNAS$_+$+ consistently outperform hand-designed cold-start techniques for recommending items for new users and predicting associated genes for new disease. We observe that different modulation functions lead to the best performance on different datasets or under different metrics, which validates the necessity of designing the modulation structure in a data-driven way.