Data Representativeness with Hyperdimensional Computing

Increased utilization of data-driven approaches, such as artificial intelligence models, has drawn scrutiny of the datasets used to develop and evaluate such approaches. Dataset assessment methods, however, continue to rely heavily on subjective manual assessments, rendering in-depth analysis (e.g., considering different levels of subgroup intersectionality) challenging for complex datasets. In this work, we present DART (Data Representativeness): a tool for the assessment of data representativeness through hyperdimensional encoding of metadata distributions. DART utilizes the principles of hyperdimensional computing to facilitate assessment of distributional similarity. This allows DART to quantitatively measure the similarity between complex metadata distributions while accurately representing different types of attributes (e.g., categorical versus numeric). The similarity measurements provided by DART serve to draw expert attention toward distributions with the highest degree of misalignment, thereby reducing the number of distributions requiring manual assessment. We demonstrate DART's utility through four case studies, each simulating a common scenario encountered in the development, evaluation, or monitoring of data-driven applications.