Systematic benchmarking of dorado basecalling models for RNA modification detection with highly multiplexed nanopore...

Area of Science:

• Molecular Biology
• Genomics
• Bioinformatics

Background:

• Nanopore direct RNA sequencing offers insights into the epitranscriptome.
• Oxford Nanopore Technologies developed basecalling models for N6-methyladenosine (m6A), inosine (I), pseudouridine (Ψ), and 5-methylcytosine (m5C).
• The performance and cross-reactivity of these models are not well understood.

Purpose of the Study:

• To systematically benchmark four modification-aware basecalling models for Nanopore direct RNA sequencing.
• To evaluate model performance on synthetic and biological samples across different species.
• To identify limitations and sources of error in current RNA modification detection methods.

Main Methods:

• Benchmarking of four modification-aware basecalling models using synthetic and in vivo rRNA samples.
• Evaluation of per-read and per-site predictions.
• Analysis of false-discovery rates and sources of false positives.
• Assessment of basecalling error- and current-based methods as alternatives.

Main Results:

• Models achieved high performance (AUC = 0.93-0.97, PR-AUC = 0.84-0.91) on balanced synthetic data.
• Performance significantly dropped on unbalanced datasets mimicking biological samples (PR-AUC: 0.04-0.09).
• High false-discovery rates (50-100%) were observed in rRNA samples.
• Cross-reactivity with other modifications and neighboring site alterations were identified as major sources of false positives.

Conclusions:

• Modification-aware basecalling models show utility but have significant limitations for RNA modification detection in complex biological samples.
• Current models exhibit high false-positive rates, necessitating careful validation.
• Basecalling error- and current-based methods provide viable alternatives for detecting RNA modifications.
• The use of control samples is crucial for mitigating false positives in epitranscriptome studies.