Benchmarking deep learning splice prediction tools using functional splice assays.

Area of Science:

• Genetics
• Molecular Biology
• Bioinformatics

Background:

• Hereditary disorders often stem from genetic variants affecting pre-messenger RNA splicing.
• Predicting the pathogenicity of variants in noncanonical splice sites (NCSS) and deep intronic (DI) regions is difficult.
• Advancements in deep learning have led to new splice prediction tools.

Purpose of the Study:

• To benchmark established and deep learning-based splice prediction tools.
• To evaluate tool performance on known variants in ABCA4 and MYBPC3 genes.
• To compare prediction accuracy with functional splice assays.

Main Methods:

• Benchmarking of ten splice prediction tools: CADD, DSSP, GeneSplicer, MaxEntScan, MMSplice, NNSPLICE, SPIDEX, SpliceAI, SpliceRover, and SpliceSiteFinder-like.
• Utilized gold standard variant sets for ABCA4 (71 NCSS, 81 DI) and MYBPC3 (61 NCSS).
• Validated predictions using midigene and minigene splice assays.

Main Results:

• SpliceRover demonstrated the best performance for ABCA4 NCSS variants.
• SpliceAI was the top performer for ABCA4 DI variants.
• An Alamut 3/4 consensus approach (GeneSplicer, MaxEntScan, NNSPLICE, SpliceSiteFinder-like) excelled for MYBPC3 NCSS variants.
• Area under the receiver operator curve (AUC) was used for performance evaluation.

Conclusions:

• Specific tools show superior performance for distinct variant types and genes.
• Clinical utility of these tools may be more limited than initially reported.
• Further refinement of splice prediction models is needed for accurate clinical application.