Recent Trend in Artificial Intelligence-Assisted Biomedical Publishing: A Quantitative Bibliometric Analysis.

Area of Science:

• Biomedical publishing
• Artificial intelligence applications
• Scientific communication

Background:

• Rapid advancements in artificial intelligence (AI) have led to its integration into various fields, including biomedical publishing.
• The extent of AI's contribution to the development of biomedical literature remains unclear.
• Understanding AI's role is crucial for maintaining scientific integrity and identifying emerging trends.

Purpose of the Study:

• To identify and quantify trends in AI-generated content within peer-reviewed biomedical literature.
• To evaluate the effectiveness of AI-detection software for analyzing biomedical abstracts.
• To assess the prevalence of AI-assisted publishing over a defined period.

Main Methods:

• Validated commercially available AI-detection software for sensitivity and specificity.
• Conducted a MEDLINE search for randomized controlled trials published between January 2020 and March 2023.
• Randomly selected 30 abstracts per quarter and analyzed them for AI-generated content probability.

Main Results:

• AI-detection software demonstrated high accuracy (100% sensitivity, 95% specificity, 97.6% AUC).
• The prevalence of abstracts with a high probability (≥90%) of AI-generated text increased from 21.7% to 36.7% during the study period (p=0.01).
• This increasing trend was consistent across various AI probability thresholds and sensitivity analyses (all p≤0.01).

Conclusions:

• The prevalence of AI-assisted publishing in peer-reviewed biomedical journals has significantly increased.
• This trend was evident even before the widespread adoption of advanced AI tools like ChatGPT.
• Further research is needed to understand factors influencing AI detection scores, such as author writing styles and AI integration during publication.