Evaluating the Group Detection Performance: The GRODE Metrics

The detection of groups of individuals is attracting the attention of many researchers in diverse fields, from automated surveillance to human-computer interaction, with a growing number of approaches published every year. Unexpectedly, the evaluation metrics for this problem are not consolidated, with some measures inherited from the people detection field, other from clustering, other designed specifically for a particular approach, thus lacking in generalization and making the comparisons between different approaches hard to be carried out. Moreover, most of the existent metrics are scarcely expressive, addressing groups as they are atomic entities, ignoring that they may have different cardinalities, and that group detection approaches may fail in capturing the exact number of individuals that compose it. This paper fills this gap presenting the GROup DEtection (GRODE) metrics, which formally define precision and recall on the groups, including the group cardinality as a variable. This gives the possibility to investigate aspects never considered so far, such as the tendency of a method of over- or under-segmenting, or of better dealing with specific group cardinalities. The GRODE metrics have been evaluated first on controlled scenarios, where the differences with alternative metrics are evident. Then, the metrics have been applied to eight approaches of group detection, on eight public datasets, providing a fresh-new panorama of the state-of-the-art, discovering interesting strengths and pitfalls of the recent approaches.