InfectA-Chat, an Arabic Large Language Model for Infectious Diseases: Comparative Analysis.

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Public Health Informatics

Background:

• Infectious diseases pose a significant global public health challenge, necessitating robust monitoring systems.
• Existing surveillance platforms predominantly use English, creating accessibility barriers for non-English speakers, particularly in regions like the Middle East.
• Middle East respiratory syndrome coronavirus (MERS-CoV) outbreaks highlight the need for localized, accessible disease monitoring tools.

Purpose of the Study:

• To introduce InfectA-Chat, a novel large language model (LLM) designed for bilingual (Arabic/English) question-and-answer tasks on infectious diseases.
• To enhance global public health efforts and individual understanding by overcoming language barriers in disease surveillance.

Main Methods:

• Instruction tuning of AceGPT-7B and AceGPT-7B-Chat models using a 55,400-entry Arabic and English domain-specific dataset.
• Performance evaluation via GPT-4 assessment on 2,770 domain-specific instruction-following data points.
• Comparative analysis against leading Arabic LLMs and state-of-the-art models (e.g., Jais-13B-Chat, Gemini, GPT-4).
• Integration of retrieval-augmented generation (RAG) for real-time data updates without retraining.

Main Results:

• InfectA-Chat demonstrated strong performance in infectious disease Q&A, validated by GPT-4.
• The model outperformed existing Arabic LLMs, including AceGPT-7B-Chat (by 43.52%) and Jais-13B-Chat (by 48.61%).
• InfectA-Chat achieved competitive results against state-of-the-art models, showing a leading performance of 23.78% against GPT-4.
• The RAG method significantly enhanced document retrieval accuracy, with improved results at higher top-k parameter values.

Conclusions:

• General Arabic LLMs exhibit limitations in providing current infectious disease information.
• InfectA-Chat offers a valuable bilingual solution to empower individuals and public health initiatives in infectious disease monitoring.
• The study underscores the importance of developing specialized AI tools to address specific public health information needs and language disparities.