Facilitating Aboriginal Perinatal Mental Health Information Access with a Retrieval-Augmented LLM-based Chatbot.

Area of Science:

• Perinatal Mental Health
• Artificial Intelligence in Healthcare
• Indigenous Health

Background:

• Aboriginal pregnant women and new mothers experience higher risks of mental health issues due to historical trauma.
• Non-Aboriginal clinicians face cultural knowledge barriers in providing sensitive perinatal care.
• Existing training methods are often insufficient to address these complex needs.

Purpose of the Study:

• To develop and evaluate a specialized Large Language Model (LLM)-based chatbot to improve question-answering for Aboriginal perinatal mental health.
• To address limitations of standard LLMs, such as hallucinations and domain-specific inaccuracies.
• To enhance clinicians' self-efficacy and support self-directed learning in culturally sensitive perinatal care.

Main Methods:

• Development of a specialized chatbot integrating Retrieval-Augmented Generation (RAG) with a semantic search engine.
• Utilizing verified external knowledge to ensure accurate and contextually relevant responses.
• Performance evaluation against a baseline GPT-3.5-turbo model and comparison of different RAG techniques.

Main Results:

• The specialized RAG LLM-based chatbot demonstrated potential for improved domain-specific question-answering.
• Accuracy and reliability were assessed in comparison to standard LLM approaches.
• The RAG integration mitigated issues like unverifiable references and hallucinatory responses.

Conclusions:

• Specialized RAG LLM chatbots can provide clinically relevant, on-demand support for healthcare providers.
• This technology has the potential to bridge knowledge gaps in Aboriginal perinatal mental health care.
• The chatbot can complement existing training, supporting self-directed learning and improving clinician confidence.