Extracting user profile via large language models and ontologies

Area of Science:

• Natural Language Processing
• Artificial Intelligence
• Computational Linguistics

Background:

• User profile extraction is crucial for personalized systems like recommendation engines and chatbots.
• Existing methods primarily leverage large language models (LLMs) on resource-rich languages.
• Less-resourced languages, such as Persian, present challenges due to limited data and complex linguistic structures.

Purpose of the Study:

• To develop and evaluate a novel multi-step approach for user profile extraction from Persian dialogue systems.
• To address the limitations of current LLMs in handling less-resourced languages for profile extraction.
• To improve the accuracy and consistency detection of extracted user profiles.

Main Methods:

• A hybrid method combining slot filling, in-context learning, and ontology-based inference was proposed.
• Extensive experiments were conducted on various models using Persian dialogue data.
• Performance was evaluated against state-of-the-art LLMs, including GPT-4o and Llama-3-70B.

Main Results:

• The proposed hybrid method achieved a 90.46 F-score for profile extraction, surpassing GPT-4o and Llama-3-70B.
• The system demonstrated superior performance in detecting information inconsistencies, achieving 92% accuracy.
• The method significantly outperformed leading LLMs in both extraction accuracy and inconsistency detection.

Conclusions:

• The developed hybrid approach effectively overcomes the challenges of user profile extraction in less-resourced languages like Persian.
• This method offers a substantial improvement over current LLM-based techniques for dialogue system applications.
• The findings highlight the potential of hybrid methods for enhancing NLP tasks in linguistically complex and data-scarce environments.