The TRIPOD-LLM reporting guideline for studies using large language models

Area of Science:

• Biomedical Informatics
• Artificial Intelligence in Healthcare
• Clinical Research Methodology

Background:

• Large language models (LLMs) are increasingly used in healthcare, but lack standardized reporting guidelines.
• Existing reporting standards need adaptation to address the unique challenges posed by LLMs in biomedical applications.

Purpose of the Study:

• To introduce TRIPOD-LLM (Transparent Reporting of a Multivariable model for Individual Prognosis or Diagnosis-Large Language Models), an extension of TRIPOD+AI.
• To provide a comprehensive checklist for transparently reporting LLM-based research in healthcare.

Main Methods:

• Development of TRIPOD-LLM through an expedited Delphi process and expert consensus.
• Creation of a modular checklist with 19 main items and 50 subitems, adaptable to various LLM research designs.
• Introduction of an interactive website for guideline completion and PDF generation.

Main Results:

• TRIPOD-LLM offers a detailed framework covering all research aspects from title to discussion.
• The guidelines emphasize transparency, human oversight, and task-specific performance reporting for LLMs.
• A modular format ensures applicability across diverse LLM research tasks and designs.

Conclusions:

• TRIPOD-LLM provides essential guidelines to enhance the quality, reproducibility, and clinical applicability of LLM research in healthcare.
• The guidelines serve as a living document, poised to evolve with advancements in the field.
• Standardized reporting through TRIPOD-LLM is crucial for the responsible adoption of LLMs in clinical practice.