Assessing clinical decision support system tools in precision oncology: piloting ring testing

Area of Science:

• Oncology
• Genomics
• Bioinformatics

Background:

• Precision cancer medicine relies on accurate molecular diagnostics and clinical decision support systems (CDSS).
• External quality assessment (EQA) is crucial for laboratory tests but lacking for CDSS tools.
• The EU4Health project PCM4EU initiative aims to enhance cancer patient survival and quality of life.

Purpose of the Study:

• To pilot an international ring test for evaluating CDSS performance in molecular cancer diagnostics.
• To document CDSS usage, performance, and manual interpretation in a real-world setting.
• To identify needs for structured EQA schemes for CDSS tools.

Main Methods:

• Generated 20 synthetic datasets mimicking variant call sets from a 500-gene panel.
• Distributed datasets to 8 laboratories across 7 countries for analysis.
• Collected results on variant interpretation and clinical significance using structured forms.

Main Results:

• Good agreement was observed for hotspot oncogenic variants across different CDSS tools.
• Variability was noted in the reporting of specific variants and their clinical interpretation.
• Eight laboratories participated, with 10 tumour-only cases analyzed.

Conclusions:

• Pilot ring test revealed clinically relevant discrepancies in CDSS interpretation.
• Emphasized the necessity for structured EQA schemes for CDSS tools.
• Identified challenges in synthetic data generation, reporting formats, and ground truth definition.