Novel evaluation approach for molecular signature-based deconvolution methods.

Area of Science:

• Oncology
• Immunology
• Bioinformatics

Background:

• The tumoral immune microenvironment (TIME) is critical for cancer prognosis and treatment.
• Computational deconvolution methods (DM) using molecular signatures (MS) analyze TIME from RNA-seq data.
• Existing benchmarking metrics inadequately assess DM performance, missing bias and accuracy analysis.

Purpose of the Study:

• To introduce a novel protocol for rigorously evaluating deconvolution methods and molecular signatures.
• To assess cell type identification accuracy and proportion prediction in TIME deconvolution.
• To benchmark state-of-the-art DMs and MSs for TIME analysis.

Main Methods:

• Developed a four-test protocol incorporating F1-score, distance to optimal point, error rates, and Bland-Altman analysis.
• Applied the protocol to benchmark six leading DMs (CIBERSORTx, DCQ, DeconRNASeq, EPIC, MIXTURE, quanTIseq).
• Utilized five murine tissue-specific MSs for comprehensive method evaluation.

Main Results:

• The novel protocol effectively evaluates cell type identification and proportion prediction accuracy.
• Benchmarking revealed systematic overestimation of cell types across most tested DM-MS pairs.
• Identified limitations in current metrics for assessing deconvolution method performance.

Conclusions:

• The developed protocol provides a robust framework for validating TIME deconvolution tools.
• Current deconvolution methods may overestimate immune cell infiltration, impacting TIME interpretation.
• Accurate TIME characterization requires improved deconvolution strategies and rigorous validation.