Reporting Guideline for Chatbot Health Advice Studies: The CHART Statement.

The rapid integration of conversational agents into medical contexts has created a surge in literature evaluating their utility for Chatbot Health Advice (CHA) across diverse clinical domains. Prior research has shown that the proliferation of these tools often outpaces the development of rigorous evaluation frameworks, leading to a landscape of inconsistent data. Scientific literature currently exhibits significant heterogeneity in how these digital interventions are documented and assessed, which complicates the ability of clinicians to synthesize findings across different platforms or models. Standardized reporting is essential to ensure that the evidence generated by these investigations remains interpretable, reproducible, and useful for the broader medical community. Existing guidelines for clinical trials or observational studies do not fully capture the nuances of algorithmic prompting, model versioning, or the specific challenges of generative artificial intelligence. This absence of evidence motivated the creation of a specialized framework to address the unique methodological requirements of automated health counseling and ensure high-quality evidence generation.

This research establishes the Chatbot Health Advice Reporting Training (CHART) statement to standardize the documentation of Generative Artificial Intelligence (AI) performance when delivering medical information. The project addresses the urgent need for transparency when chatbots summarize clinical evidence for patient or provider use, ensuring that the underlying data sources are clearly identified. Investigators sought to create a comprehensive checklist that covers every phase of a study from title selection and abstract formatting to data availability and ethical disclosures. The initiative aims to assist peer reviewers and editors in identifying high-quality research within the burgeoning field of digital health while providing authors with a clear roadmap. By defining specific subitems for model identifiers and query strategies, the study provides a framework for future experimental designs that prioritize technical reproducibility. The framework targets the reduction of reporting bias and the improvement of cross-study comparability across diverse clinical domains and geographic regions. Developers intended for this tool to serve as a foundational resource for the international multidisciplinary community involved in the intersection of computer science and clinical medicine.

The development process began with a comprehensive systematic review designed to identify existing variations in conduct, reporting, and methodology across the current chatbot health advice literature. Researchers synthesized these findings into an initial draft checklist containing potential reporting requirements that addressed the specific technical and clinical nuances of generative artificial intelligence. An international multidisciplinary modified asynchronous Delphi consensus process then engaged 531 diverse stakeholders to refine the items and ensure broad applicability across different healthcare systems. Three synchronous panel consensus meetings involving 48 key stakeholders followed this phase to resolve remaining points of contention and finalize the core components of the framework. The team subjected the resulting draft to rigorous pilot testing to ensure practical utility, clarity, and ease of use for authors during the manuscript preparation process. Methodologists incorporated feedback from clinicians, researchers, and editors to finalize the 12 primary items and 39 subitems that constitute the complete CHART statement.

The CHART statement comprises a structured checklist of 12 main items and 39 detailed subitems designed to promote transparent and comprehensive reporting of chatbot health advice studies. Specific requirements for model identifiers in subitem 3ab and model details and configuration in subitems 4abc ensure that the exact version of the AI is documented for future replication. The guideline mandates the disclosure of prompt engineering in subitems 5ab and query strategy in subitems 6abcd to allow for a technical understanding of how the chatbot was interrogated. Performance evaluation in subitems 7ab and data analysis in subitem 9a provide clear instructions on how to present algorithmic accuracy and the statistical significance of the findings. Administrative items such as funding in subitem 12b, ethics in subitem 12c, and protocol in subitem 12d are integrated to uphold the highest standards of scientific integrity. The consensus process successfully harmonized the perspectives of 531 international participants into a single cohesive reporting standard that covers the entire research lifecycle from title to data availability. Results indicate that the checklist covers every critical reporting domain, including title (1a), abstract (1b), and results (10abc).

Implementing the CHART statement will likely enhance the clarity and interpretability of chatbot health advice studies, providing a vital tool for clinicians and researchers to evaluate AI-generated medical summaries. Adherence to these standards will facilitate more robust meta-analyses by reducing the heterogeneity currently found in the literature and allowing for more direct comparisons between different chatbot models. The guideline supports the ethical deployment of generative artificial intelligence by requiring transparent disclosures, ethical approvals, and the public registration of study protocols. Future research in digital health communication should adopt these reporting items to ensure findings are accessible, reproducible, and trustworthy for the global scientific community. The methodological diagram serves as a practical aid for authors during the manuscript preparation phase, helping to visualize the recommended workflow for high-quality reporting. Every disease, condition, or application mentioned in future studies must now be reported with the transparency required to support key stakeholders including editors and readers. Ultimately, this framework strengthens the evidence base for using automated systems in clinical evidence synthesis and health advice provision while fostering innovation in the field.