Semblance: An empirical similarity kernel on probability spaces.

Area of Science:

• Data Science
• Machine Learning
• Statistical Modeling

Background:

• Determining observation proximity is crucial for data science tasks like clustering and classification.
• Existing similarity measures often rely on arbitrary functions when data distributions are unknown.
• This limitation hinders robust analysis, especially with complex or uncharacterized datasets.

Purpose of the Study:

• Introduce Semblance, a novel, distribution-free proximity measure for data.
• To develop a similarity function that leverages empirical data distributions.
• To enhance the analysis of observations, particularly those at the distribution's outskirts.

Main Methods:

• Defined Semblance based on the empirical distribution of features for pair-wise similarity.
• Validated Semblance as a Mercer kernel for kernel-based learning algorithms.
• Applied Semblance to diverse data modalities, including single-cell transcriptomics, image reconstruction, and financial forecasting.

Main Results:

• Semblance demonstrated improved performance over conventional methods in simulations.
• Real-world case studies confirmed Semblance's effectiveness in diverse applications.
• The distribution-free nature and focus on outliers provide distinct analytical advantages.

Conclusions:

• Semblance offers a principled and versatile approach to measuring proximity in data science.
• Its ability to handle unknown distributions and emphasize outliers enhances downstream analyses.
• Semblance represents a significant advancement for machine learning and statistical modeling across various scientific domains.