[Can large language models answer clinical questions?]

Area of Science:

• Artificial Intelligence in Medicine
• Natural Language Processing
• Clinical Decision Support

Context:

• The rapid advancement of large language models (LLMs) like ChatGPT presents new opportunities for medical applications.
• Existing research indicates LLMs can provide accurate information on disease management and treatment.
• However, the critical need for alignment with international medical guidelines is paramount.

Purpose:

• To evaluate the accuracy, readability, thoroughness, and guideline consistency of LLM responses in a medical context.
• To highlight the necessity for relevance, pertinence, and up-to-date sourcing in LLM-generated medical information.
• To identify research gaps concerning cross-LLM consistency, multilingual capabilities, and reliability for complex cases.

Summary:

• LLMs demonstrate potential in medicine, offering correct answers for disease management and treatment queries.
• Key evaluation criteria include accuracy, readability, thoroughness, and adherence to international guidelines.
• Ensuring source relevance, currency, and consistency across different LLMs and languages is essential for clinical utility.

Impact:

• LLMs can potentially enhance medical education and support clinical decision-making processes.
• Further validation and comparative studies against established international guidelines are required before widespread clinical integration.
• Addressing reliability for rare diseases and improving training methodologies will enhance LLM trustworthiness in healthcare.