Target-Oriented Multi-Granular Semantic-Aware Network for Source-Free Domain Adaptation

Source-free Domain Adaptation (SFDA) realizes cross-domain transfer by adapting a pretrained source model to an unlabeled target domain without accessing data in the source domain. Existing SFDA methods mainly develop subdomain-distribution-oriented framework to extract the fine-grained communal semantics from the target domain. However, they may neglect the facts that (a) smaller knowledge subsets exist within subcategories and overlaps may occur among some of these subsets; and (b) an object may appear at different granularities in the source and target domains so the semantics between them cannot be fully aligned. To this end, this study proposes a Target-oriented Multi-granular Semantic-aware Network (TMSN) for SFDA, including (a) Semantic-aware Micro-community Learning (SML) that introduces micro-communities as small-scale knowledge subsets, employs a learnable semantic-aware module and a semantic-promotability-based segmentation mechanism to evaluate the assignment of target samples to micro-communities, and establishes a multi-scale mapping from the target set into the micro-communities based on the learned assignment patterns; and (b) Target-oriented Semantic Promotability Enhancement (TSPE) that leverages semantic promotability to allocate features from the target domain to micro-communities in the same superordinate unit, and push away those in the different superordinate unit. In this way, the overlapping and asymmetrical relationships across micro-communities are captured, while the multi-granular semantics is learned. Building on the micro-community, we formally show the feasibility of SML and TSPE in batch training, and extensive experiments conducted in close-set and partial-set SFDA scenarios and in object detection demonstrate the superiority of TMSN. Source codes are available at https://github.com/zzf495/TMSN.