Advancing codon language modeling with synonymous codon constrained masking.

Area of Science:

• Bioinformatics
• Computational Biology
• Genomics

Background:

• Current codon language models often mix codon usage with amino acid semantics, hindering DNA-level biological insights.
• Existing models struggle to capture nucleotide-specific patterns due to conflated semantic information.

Purpose of the Study:

• To introduce SynCodonLM, a novel codon language model designed to disentangle codon-level and protein-level semantics.
• To enable the learning of nucleotide-specific patterns by enforcing biologically grounded constraints.

Main Methods:

• Developed SynCodonLM, a codon language model with a constraint ensuring masked codons are predicted only from synonymous options, guided by the protein sequence.
• Implemented a masking strategy that excludes non-synonymous codons from the prediction space before softmax.
• Modified clustering to group codons by nucleotide properties instead of amino acid identity.

Main Results:

• SynCodonLM successfully disentangles codon and protein semantics, learning nucleotide-specific patterns.
• The model reveals biological structure aligned with DNA-level properties through its clustering approach.
• SynCodonLM outperformed existing models on six out of seven benchmarks sensitive to DNA-level features, including mRNA and protein expression.

Conclusions:

• SynCodonLM represents an advancement in domain-specific representation learning for DNA sequences.
• The model offers new possibilities for sequence design in synthetic biology and deeper exploration of bioprocesses.
• This approach enhances the ability to model DNA sequences by respecting biological constraints.