Improving Dietary Supplement Information Retrieval: Development of a Retrieval-Augmented Generation System With Large...

Area of Science:

• Biomedical Informatics
• Artificial Intelligence in Healthcare
• Dietary Supplement Research

Background:

• Dietary supplements (DSs) face challenges with misinformation and varying efficacy due to less stringent regulations than pharmaceuticals.
• Accurate DS information is crucial for informed decision-making by consumers and healthcare professionals.

Purpose of the Study:

• To enhance dietary supplement (DS) question-answering systems by integrating the iDISK2.0 knowledge base with a retrieval-augmented generation (RAG) system.
• To improve the accuracy and reliability of DS information retrieval and reduce instances of AI hallucination.

Main Methods:

• Developed iDISK2.0 by integrating data from authoritative sources and applying data cleaning techniques.
• Implemented a RAG system combining a biomedical knowledge graph with large language models (LLMs) for context-aware information retrieval.
• Evaluated system performance using true-or-false and multiple-choice questions on DS effectiveness and drug interactions.

Main Results:

• The RAG system achieved 99% accuracy on DS effectiveness and 95% on DS-drug interactions, significantly outperforming stand-alone LLMs.
• iDISK2.0 integrates over 174,000 entities, including ingredients, products, diseases, and drugs, with defined relationships.
• The system provided accurate, evidence-based responses through a user-friendly interface, minimizing data noise.

Conclusions:

• Integrating a knowledge graph with RAG and LLM technologies effectively addresses limitations of stand-alone LLMs for DS information retrieval.
• This approach enhances accuracy and reduces misinformation in health applications by combining structured data with AI.
• Future work aims to expand the framework to broader biomedical areas and refine evaluation with real-world queries.