Friend or Foe: Fine-Grained Categorization With Weak Supervision

Multi-instance learning (MIL) is widely acknowledged as a fundamental method to solve weakly supervised problems. While MIL is usually effective in standard weakly supervised object recognition tasks, in this paper, we investigate the applicability of MIL on an extreme case of weakly supervised learning on the task of fine-grained visual categorization, in which intra-class variance could be larger than inter-class due to the subtle differences between subordinate categories. For this challenging task, we propose a new method that generalizes the standard multi-instance learning framework, for which a novel multi-task co-localization algorithm is proposed to take advantage of the relationship among fine-grained categories and meanwhile performs as an effective initialization strategy for the non-convex multi-instance objective. The localization results also enable object-level domain-specific fine-tuning of deep neural networks, which significantly boosts the performance. Experimental results on three fine-grained datasets reveal the effectiveness of the proposed method, especially the importance of exploiting inter-class relationships between object categories in weakly supervised fine-grained recognition.