CLARITY: comparing heterogeneous data using dissimilarity

Area of Science:

• Multidisciplinary data integration
• Computational biology
• Quantitative social sciences

Background:

• Integrating heterogeneous datasets poses significant challenges due to variations in meaning, scale, and reliability.
• Scientific inquiry often requires assessing the conservation of entity similarities across disparate data sources.

Purpose of the Study:

• To introduce CLARITY, a novel method for quantifying consistency across datasets.
• To identify and interpret inconsistencies arising from cross-dataset comparisons.
• To provide a robust framework for analyzing relationships between different data modalities.

Main Methods:

• CLARITY employs a non-parametric approach robust to noise and scaling differences.
• It decomposes similarity matrices into 'structural' and 'relationship' components for comparison.
• Significance is assessed using dataset-appropriate resampling techniques.

Main Results:

• Demonstrated CLARITY's utility across diverse comparisons: gene methylation vs. expression, linguistic evolution, and economic vs. cultural metrics.
• Quantified cross-dataset consistency and pinpointed areas of divergence.
• The method proved effective in handling data with varying properties.

Conclusions:

• CLARITY offers a powerful, flexible tool for comparing datasets from different scientific disciplines.
• The method facilitates a deeper understanding of conserved and divergent patterns across data types.
• Available as an R package, CLARITY promotes reproducible cross-disciplinary research.