Re-evaluating retrosynthesis algorithms with Syntheseus.

Area of Science:

• Computational chemistry
• Machine learning
• Chemical synthesis

Background:

• Automated synthesis planning is a growing research area combining chemistry and machine learning.
• Existing techniques face challenges due to imperfect benchmarks and inconsistent comparisons, hindering progress.
• There is a need for standardized evaluation frameworks to accurately assess algorithm performance.

Purpose of the Study:

• To introduce SYNTHESEUS, a novel library and benchmarking framework for automated synthesis planning.
• To promote best practices and enable consistent, meaningful evaluation of synthesis planning algorithms.
• To re-evaluate existing retrosynthesis algorithms under controlled conditions.

Main Methods:

• Development of the SYNTHESEUS library, incorporating an extensive benchmarking framework.
• Implementation of default best practices for reproducible and standardized evaluations.
• Controlled experiments to re-evaluate multiple single-step and multi-step retrosynthesis algorithms.

Main Results:

• The SYNTHESEUS framework facilitates consistent and meaningful evaluation of synthesis planning algorithms.
• Re-evaluation of previous state-of-the-art models using SYNTHESEUS resulted in a changed ranking of their performance.
• Controlled evaluation experiments highlight the impact of benchmark quality on algorithm assessment.

Conclusions:

• SYNTHESEUS provides a robust platform for advancing automated synthesis planning research.
• The study underscores the critical need for improved benchmarks in this field.
• Recommendations are provided for future research, encouraging community engagement in benchmark development.