Performance assessment of computational tools to detect microsatellite instability

Area of Science:

• Genomics
• Cancer Research
• Bioinformatics

Background:

• Microsatellite instability (MSI) is a key biomarker for guiding immune checkpoint inhibitor therapy in various cancers.
• Computational tools are crucial for classifying samples as high MSI or microsatellite stable using next-generation sequencing data.
• Existing MSI tools often lack clear usage guidelines and independent performance benchmarks.

Purpose of the Study:

• To comprehensively assess and benchmark the performance of leading MSI computational tools.
• To evaluate tool performance across diverse sequencing data types, including whole exome, whole genome, gene panel, and RNA sequencing.
• To identify reliable MSI tools for clinical application and guide future development.

Main Methods:

• Evaluated eight prominent MSI prediction tools using multiple unique datasets.
• Compared tool performance on whole exome sequencing (WES), whole genome sequencing (WGS), gene panel, and RNA sequencing (RNA-Seq) data.
• Assessed performance using receiver operating characteristic (ROC) and precision-recall area under the curve (AUC) metrics, and evaluated inter-tool agreement.

Main Results:

• Most MSI tools replicated original findings on WES data but showed reduced performance on WGS data.
• Significant discrepancies were observed in tool agreement and performance on gene panel data compared to commercial software.
• Optimal threshold cut-offs for MSI classification varied substantially depending on the sequencing type.
• RNA-Seq specific MSI tools were outperformed by DNA-based tools, and even DNA tools showed decreased precision when combining diverse datasets.

Conclusions:

• The performance of MSI tools can significantly degrade when applied to datasets different from their original evaluation cohorts.
• Caution is advised when using MSI tools on WGS or RNA-Seq data, as well as on combined datasets, due to potential performance limitations.
• MSIsensor2 and MANTIS showed robust performance across most datasets but experienced reduced precision when all data were aggregated, highlighting the need for careful tool selection and validation.