The Promise and Perils of Autonomous AI in Science

AbstractIn this edition of NEJM AI, Ifargan and colleagues present data-to-paper, an autonomous platform designed to mimic human scientific practice by guiding a large language model through a stepwise research process to produce complete research papers. While the platform shows promise, significant errors frequently occur with complex datasets, requiring human intervention for correction. Here, we explore the risks of over-reliance on such automated tools, echoing potential threats to scientific integrity and a surge in low-quality publications, while also considering their potential role in the reproduction and verification of scientific findings. To address these challenges, we recommend the development of clearer guidelines and ethical standards for the use of artificial intelligence (AI) in research, fostering human-AI collaboration to enhance research quality while preserving human oversight, and integrating the innovative "data-chaining" transparency mechanisms more broadly to support reproducibility and traceability.