CodonRL: Multi-Objective Codon Sequence Optimization Using Demonstration-Guided Reinforcement Learning

Area of Science:

• Computational biology
• Synthetic biology
• Bioinformatics

Background:

• Optimizing mRNA sequences for translation efficiency, RNA stability, and compositional properties is complex due to vast search spaces and interacting objectives.
• Existing methods like dynamic programming struggle with extensibility, while deep generative models require extensive training data, and reinforcement learning faces challenges with delayed rewards and large action spaces.

Purpose of the Study:

• To develop a reinforcement learning framework, CodonRL, for efficient and customizable mRNA sequence optimization.
• To address limitations of existing methods by incorporating structural priors and enabling multi-objective trade-offs during inference.

Main Methods:

• CodonRL utilizes reinforcement learning with efficient folding feedback and demonstration-guided replay to learn structural priors for mRNA design.
• It employs LinearFold for rapid intermediate reward computation during training and ViennaRNA for final evaluation.
• Milestone-based intermediate rewards and expert sequence warm-up accelerate convergence and handle delayed feedback in long-range optimization.

Main Results:

• CodonRL demonstrated superior performance compared to GEMORNA on 55 human proteins.
• Achieved an average 9.5% higher codon adaptation index (CAI), 25.4 kcal/mol more favorable minimum free energy (MFE), and 3.4% lower uridine content.
• Improved codon stabilization coefficient (CSC) in over 90% of benchmark proteins under matched constraints.

Conclusions:

• CodonRL provides a powerful framework for designing mRNA sequences with improved translation efficiency, structural stability, and reduced immunogenicity.
• The method allows for continuous objective reweighting at inference time, offering flexibility in mRNA design.
• CodonRL represents a significant advancement in computational mRNA design, particularly in scenarios with scarce training data.