Deliberate Problem-solving with a Large Language Model as a Brainstorm Aid Using a Checklist for Prompt Generation

Area of Science:

• Artificial Intelligence
• Clinical Decision Support
• Medical Informatics

Background:

• Large language models (LLMs) are advanced AI tools that generate text using autoregression.
• Clinicians face challenges in formulating effective prompts for LLMs to aid in clinical problem-solving.
• Familiarity with generative AI technology is often limited among healthcare professionals.

Purpose of the Study:

• To explore the utility of checklists in generating optimal prompts for LLMs in a clinical context.
• To investigate the potential of LLMs as a brainstorming aid for medical problem-solving.
• To develop a systematic approach for prompt generation to improve differential diagnoses.

Main Methods:

• Systematic generation of prompts using varying checklist-based strategies.
• Application of generated prompts to selected clinical case scenarios.
• Evaluation of LLM-generated differential diagnoses for appropriateness and clinical relevance.

Main Results:

• Checklists can facilitate the creation of carefully worded queries for LLMs.
• Systematic prompting enhances the ability of LLMs to serve as a brainstorming aid.
• The study reports the creation of a potential checklist for prompt generation.

Conclusions:

• Checklists offer a structured method for clinicians to interact with LLMs.
• Leveraging LLMs with effective prompts can support clinical decision-making and differential diagnosis.
• Further development and validation of prompt generation checklists are warranted.