Beyond the Hype-The Actual Role and Risks of AI in Today's Medical Practice: Comparative-Approach Study

Area of Science:

• Artificial Intelligence in Healthcare
• Academic Writing Technologies

Background:

• Artificial intelligence (AI) is increasingly impacting healthcare, with advanced models like ChatGPT-4 and Microsoft Bing assisting in complex tasks.
• These AI tools demonstrate capabilities beyond text generation, including aiding literature searches and refining web queries for research.

Purpose of the Study:

• To evaluate the independent academic paper authoring capabilities of AI models, specifically ChatGPT-4 and Microsoft Bing.
• To assess AI performance across relevance, accuracy, clarity, and academic tone and style.
• To consider the ethical implications and practical integration of AI in academic writing.

Main Methods:

• Systematic assessment of ChatGPT-4 and Microsoft Bing for academic paper assistance in general practice.
• Evaluation criteria included relevance, accuracy, clarity, and tone/style.
• Consideration of AI's knowledge cut-off dates and search functionalities.

Main Results:

• ChatGPT-4 provided in-depth, relevant content on AI in healthcare but had a 72% citation accuracy with some fabricated references.
• Microsoft Bing offered concise overviews with 46% citation accuracy, including non-peer-reviewed sources.
• Both models produced clear, coherent text with an academic tone; ChatGPT-4 excelled in technical depth.

Conclusions:

• Neither ChatGPT-4 nor Microsoft Bing can independently author comprehensive academic papers, despite individual strengths.
• ChatGPT-4 offers superior depth but struggles with citation accuracy; Microsoft Bing is concise but lacks detail.
• Future AI development could integrate strengths for optimized academic support, emphasizing critical researcher oversight.