Related Experiment Video
Updated: Sep 12, 2025
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Development and Reliability Assessment of an Artificial Intelligence-Driven Urticaria Support (AIDUS) Chatbot
Felix Aulenbacher1, Annika Gutsche1, Benedict Bihlmaier2
1Institute of Allergology, Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany; Immunology and Allergology, Fraunhofer Institute for Translational Medicine and Pharmacology, Berlin, Germany.
A new AI chatbot, AIDUS, offers reliable, high-quality information for chronic urticaria (CU) patients and physicians. It significantly outperforms general AI models like ChatGPT in accuracy and consistency for CU-specific queries.
Area of Science:
- Medical Informatics
- Artificial Intelligence in Healthcare
- Dermatology
Background:
- Chronic urticaria (CU) significantly impacts patient quality of life.
- Patients often seek online information due to lengthy diagnosis and treatment times for CU.
- There is a need for reliable, accessible information sources for CU management.
Purpose of the Study:
- To develop an AI chatbot, AIDUS (Artificial Intelligence-Driven Urticaria Support), for patients and physicians managing CU.
- To evaluate AIDUS's reliability and accuracy in providing CU-specific information.
- To compare AIDUS's performance against ChatGPT-3.5 and ChatGPT-4o.
Main Methods:
- AIDUS was developed using JavaScript and OpenAI's ChatGPT, trained on a curated selection of peer-reviewed CU publications.
- A systematic PubMed review ensured the quality of integrated information.
- AIDUS performance was evaluated using 100 validated CU-related questions and compared to ChatGPT models.
Main Results:
- AIDUS demonstrated superior accuracy (94.6%) compared to ChatGPT-3.5 (42.6%) and ChatGPT-4o (85.7%).
- The AI chatbot showed higher consistency and stability in answering CU-specific questions.
- Optimized settings enhanced AIDUS's efficiency and accuracy.
Conclusions:
- AIDUS provides a reliable source of high-quality, evidence-based information for chronic urticaria.
- The chatbot effectively addresses the information needs of both patients and physicians.
- AIDUS serves as a valuable assistive tool, complementing professional medical consultation.
