Almost Nobody Is Using ChatGPT to Write Academic Science Papers (Yet).

Area of Science:

• Scientific publishing
• Artificial Intelligence in Research
• Academic Integrity

Background:

• The rapid advancement of generative AI, including ChatGPT, raises concerns about its potential undisclosed use in scientific writing.
• Previous methods for detecting AI-generated text often rely on obvious linguistic markers, which may not be present in sophisticated AI writing.

Purpose of the Study:

• To quantify the prevalence of undisclosed ChatGPT use in scientific paper introductions.
• To assess the extent to which generative AI is impacting mainstream scientific journals.

Main Methods:

• A stylistics approach was employed to analyze 19,000 scientific introductions from ten MDPI journals published in 2023 and early 2024.
• The methodology focused on detecting AI-generated text without relying on explicit AI-related phrases.

Main Results:

• An estimated 1-3% of the analyzed introductions showed evidence of undisclosed ChatGPT use for writing or significant editing.
• The rate of undisclosed AI writing was consistent across the selected MDPI journals.
• No significant pollution of mainstream journals by generative AI was detected.

Conclusions:

• Generative AI, specifically ChatGPT, is not yet widely adopted for late-stage writing or editing by the majority of scientists.
• The study provides the first systematic evidence of detecting undisclosed AI writing in published manuscripts, suggesting a cautious approach by researchers.
• Current AI integration into scientific publishing appears limited, with scientists largely remaining hesitant to use these tools extensively.