An AI system to help scientists write expert-level empirical software

Area of Science:

• Computational science
• Artificial intelligence
• Scientific discovery acceleration

Background:

• Scientific discovery is often slowed by manual software development for computational experiments.
• There is a need for automated systems to generate high-quality scientific software.

Purpose of the Study:

• To introduce Empirical Research Assistance (ERA), an AI system designed to create expert-level scientific software.
• To demonstrate ERA's capability in systematically improving quality metrics and navigating complex solution spaces.
• To showcase ERA's effectiveness across diverse scientific domains.

Main Methods:

• ERA utilizes a Large Language Model (LLM) integrated with Tree Search (TS).
• The system explores and integrates external research ideas to enhance software quality.
• Quality metrics are systematically improved through AI-driven exploration and optimization.

Main Results:

• ERA discovered 40 novel bioinformatics methods for single-cell data analysis, surpassing human-developed methods.
• ERA generated 14 epidemiological models for COVID-19 hospitalization forecasting, outperforming the CDC ensemble.
• Expert-level software was produced for geospatial analysis, neural activity prediction, and numerical integration.

Conclusions:

• ERA demonstrates expert-level performance in generating scientific software across various disciplines.
• The AI system significantly accelerates scientific progress by automating software creation.
• ERA's approach represents a substantial advancement in computational research assistance.