ChatGPT: how to use it and the pitfalls/cautions in academia

Area of Science:

• Artificial Intelligence in Scientific Research
• Medical Research Tools
• Academic Research Methodologies

Background:

• Large Language Models (LLMs) are transforming academic research.
• LLMs like ChatGPT are increasingly used in medical and scientific fields.
• Understanding their capabilities and limitations is crucial for researchers.

Purpose of the Study:

• To review the current state, applications, and limitations of LLMs in medical and scientific research.
• To evaluate the performance of commercial LLM services (GPT-4o, Claude 3, Gemini Pro) and specialized research platforms.
• To identify best practices for integrating LLMs into the research process.

Main Methods:

• Systematic review of recent literature and AI development reports.
• Evaluation of LLM performance, applications, and limitations across research stages.
• Comparative analysis of commercial LLMs and specialized research platforms.

Main Results:

• LLMs show potential for improving research efficiency in literature review, data analysis, and manuscript preparation.
• GPT-4o and o1 excel overall, Claude 3 in writing/coding, Gemini Pro in multimodal processing.
• Retrieval-Augmented Generation models and specialized tools enhance accuracy and information access.

Conclusions:

• LLMs are valuable complementary tools for research productivity, not primary drivers.
• Careful selection based on model strengths is essential.
• Addressing hallucination risks, biases, plagiarism, and privacy concerns is critical.
• Successful LLM integration requires understanding limitations, adhering to ethics, and preserving researcher autonomy.