Chatbot for the Return of Positive Genetic Screening Results for Hereditary Cancer Syndromes: Prompt Engineering...

Area of Science:

• Genomic Medicine
• Artificial Intelligence in Healthcare
• Bioinformatics and Computational Biology

Background:

• Growing demand for population-wide genomic screening strains genetic counseling resources.
• Large language models (LLMs) show promise for genomic services, but their use in returning positive results is underexplored.
• Innovative delivery models are needed to address resource limitations in genomic healthcare.

Purpose of the Study:

• To design, implement, and evaluate a GPT-4 powered chatbot for returning positive genomic screening results.
• To assess the chatbot's effectiveness in communicating complex genetic information within the In Our DNA SC program.
• To explore the potential of LLMs to enhance accessibility in genomic service delivery.

Main Methods:

• A 3-step prompt engineering process using retrieval-augmented generation and few-shot techniques was employed.
• Chatbot training utilized patient FAQs, genetic counseling scripts, and patient-derived queries.
• Performance was evaluated by domain experts rating responses to hypothetical patient scenarios on an 8-criteria Likert scale.

Main Results:

• The chatbot achieved an average expert rating of 3.86 out of 5 across all criteria.
• Highest scores were for tone (4.25) and usability (4.25), indicating effective and user-friendly communication.
• Program accuracy received the lowest score (3.25), suggesting a need for enhanced program-specific details.

Conclusions:

• LLM-powered chatbots are feasible for supporting the return of positive genomic screening results.
• The developed chatbot effectively handled queries, maintained boundaries, and provided user-friendly responses.
• Future work will focus on hybrid models to improve scalability, accuracy, and accessibility in genomic healthcare delivery.