Ordinal Characterization of Similarity Judgments

Characterizing judgments of similarity within a perceptual or semantic domain, and making inferences about the underlying structure of this domain from these judgments, has an increasingly important role in cognitive and systems neuroscience. We present a new framework for this purpose that makes limited assumptions about how perceptual distances are converted into similarity judgments. The approach starts from a dataset of empirical judgments of relative similarities: the fraction of times that a subject chooses one of two comparison stimuli to be more similar to a reference stimulus. These empirical judgments provide Bayesian estimates of underling choice probabilities. From these estimates, we derive indices that characterize the set of judgments in three ways: compatibility with a symmetric dis-similarity, compatibility with an ultrametric space, and compatibility with an additive tree. Each of the indices is derived from rank-order relationships among the choice probabilities that, as we show, are necessary and sufficient for local consistency with the three respective characteristics. We illustrate this approach with simulations and example psychophysical datasets of dis-similarity judgments in several visual domains and provide code that implements the analyses at https://github.com/jvlab/simrank.