VLExpan: A visual-enhanced LLM framework with inductive and deductive policies for entity set expansion

Entity Set Expansion (ESE) is a promising knowledge-acquisition task that aims to retrieve the entities sharing the same semantic class with a small seed entity set. Most existing methods employ a bootstrap framework to iteratively expand the seed entities based on a given corpus. However, these methods mainly focus on the textual information, which limits the model's ability to perform fine-grained ESE and recall long-tail positive entities. In addition, the bootstrap framework suffers from the issue of error propagation. Hence, in this paper, we propose a Visual-enhanced LLM framework with inductive and deductive policies (VLExpan). Firstly, we introduce the visual information and iteratively expand the seed entities with a vision-language model. Secondly, we utilize the LLM to induce the class name of seed entities. Finally, we employ a deductive policy to refine the previous expansion with the class name and LLM. To evaluate the effectiveness of VLExpan, we conduct extensive experiments on a public dataset SE2 and our constructed dataset NERD-Img. Our method improves the average score of MAP@10, MAP@20 and MAP@50 by 3.36 % and 4.51 % respectively. The dataset and source code of this paper are available at https://github.com/Delicate2000/VLExpan.