Assessing clinical decision support system tools in precision oncology: piloting ring testing

Area of Science:

• Oncology
• Genomics
• Bioinformatics

Background:

• Precision cancer medicine aims to improve patient survival and quality of life.
• High-quality molecular cancer diagnostics are essential for effective precision medicine.
• Clinical decision support systems (CDSS) are increasingly vital for interpreting comprehensive genomic profiling data.

Purpose of the Study:

• To pilot an international ring test for evaluating the performance and interpretation of CDSS tools in precision cancer medicine.
• To address the lack of external quality assessment schemes for CDSS, contrasting with mandatory schemes for laboratory tests.

Main Methods:

• Generated 20 synthetic datasets mimicking variant call sets from a 500-gene panel across various cancer types.
• Eight laboratories from seven countries participated in the ring test.
• Participants analyzed tumour-only and tumour-normal cases using CDSS tools and submitted structured results.

Main Results:

• Good agreement was observed for oncogenic hotspot variants across different CDSS tools.
• Significant variability was found in the reporting of variants and their clinical interpretation.
• Eight laboratories participated, submitting results for 10 tumour-only cases.

Conclusions:

• The pilot ring test identified clinically relevant discrepancies in CDSS interpretation, emphasizing the need for external quality assessment.
• Challenges were noted in generating realistic synthetic data, designing reporting formats, defining ground truth, and manual interpretation.