Generative AI in peer review process for occupational health

Area of Science:

• Occupational Health
• Academic Publishing
• Artificial Intelligence

Background:

• Generative Artificial Intelligence (AI) tools can significantly improve academic manuscript proofing.
• This study quantifies the effectiveness and efficiency of AI in manuscript review for the first time.

Purpose of the Study:

• To evaluate the effectiveness and efficiency of Generative AI tools in providing feedback and recommendations on occupational health manuscript drafts.
• Comparison of AI tools (Microsoft Copilot, ChatGPT, Google Gemini 1.0) against human reviewers.

Main Methods:

• Manuscripts were reviewed by AI tools and human reviewers in February 2024.
• Feedback and recommendations were scored by expert panels on relevance, completeness, accuracy, and constructiveness.
• Efficiency was measured by review time; statistical analysis was performed.

Main Results:

• Generative AI tools were significantly more effective in providing feedback (P < 0.001).
• Human reviewers were significantly better at providing recommendations (P < 0.01).
• AI tools demonstrated superior time-efficiency but required output verification for accuracy.

Conclusions:

• Generative AI can enhance manuscript speed and accuracy, improving knowledge dissemination in occupational health.
• AI integration offers potential for greater efficiencies in the publication process.