Model-based evaluation of spatiotemporal data reduction methods with unknown ground truth through optimal...

Area of Science:

• Computational biology
• Data science
• Bioinformatics

Background:

• Benchmarking data reduction methods for dynamic or spatial visualization and interpretation (DSVI) is challenging due to data complexity, lack of ground truth, and time-dependent metrics.
• Existing studies often rely on independent static visualization or interpretability metrics requiring ground truth, limiting comprehensive evaluation.

Purpose of the Study:

• To propose the MIBCOVIS framework, a comprehensive and interpretable computational approach for benchmarking DSVI methods.
• To enhance visualization and interpretability of high-dimensional data without ground truth, integrating robust metrics into a semi-supervised hierarchical Bayesian model.

Main Methods:

• Developed the MIBCOVIS framework, integrating five metrics, including a novel time-ordered Markov-based structural metric.
• Applied MIBCOVIS to linear and nonlinear dimensionality reduction methods across CyTOF, scRNA-seq, and CODEX data for dynamic and spatial biological processes.
• Assessed method accuracy and interaction effects, comparing accuracy distributions rather than single scores.

Main Results:

• MIBCOVIS enables joint evaluation of visualization and interpretability, outperforming single-metric approaches.
• Prioritizing average performance can mask method-specific strengths and weaknesses.
• Identified optimal parameters and recommended methods, such as the optimized variational contractive autoencoder, for specific data complexities.

Conclusions:

• The MIBCOVIS framework provides a robust approach for evaluating DSVI methods across diverse biological data.
• Findings highlight the importance of considering data complexity and joint metric evaluation for optimal DSVI.
• MIBCOVIS shows significant potential for advancing the analysis of dynamic single-cell atlases and spatiotemporal data reduction models.