Applications of the Natural Language Processing Tool ChatGPT in Clinical Practice: Comparative Study and Augmented...

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Clinical Research Methodology

Background:

• Comparative analysis of human researchers and OpenAI's ChatGPT in systematic review tasks.
• Assessment of natural language processing (NLP) model applications in clinical practice through a review of 5 studies.

Purpose of the Study:

• Evaluate reliability between ChatGPT and human researchers in extracting key information from clinical articles.
• Investigate practical uses of NLP in clinical settings via selected studies.

Main Methods:

• Systematic review of clinical articles independently performed by human researchers and ChatGPT.
• Agreement assessment using Fleiss and Cohen κ statistics for parameter extraction.

Main Results:

• High concordance between ChatGPT and human researchers for most parameters, with lower agreement on study design, clinical task, and implementation.
• Identified 5 studies showcasing diverse NLP applications in clinical settings, improving efficiency and outcomes.
• Examples include allergy detection, classification, and psychotherapy quality metrics for veterans.

Conclusions:

• NLP models, including ChatGPT, show potential for systematic reviews and clinical tasks, enhancing healthcare efficiency and accuracy.
• Limitations exist, but NLP offers a promising avenue for improving healthcare.
• Future research should expand clinical applications and address ethical considerations.