One Metric to Measure Them All: Localisation Recall Precision (LRP) for Evaluating Visual Detection Tasks

Area of Science:

• Computer Vision
• Machine Learning
• Image Analysis

Background:

• Average Precision (AP) is a common but limited metric for visual detection, lacking interpretability and localization quality assessment.
• Panoptic Quality (PQ) addresses some AP limitations but is restricted to panoptic segmentation tasks.

Purpose of the Study:

• To propose Localisation Recall Precision (LRP) Error as a comprehensive performance metric for diverse visual detection tasks.
• To introduce Optimal LRP (oLRP) Error for evaluating detectors and determining optimal deployment thresholds.

Main Methods:

• LRP Error computes average matching error based on localization and classification quality for a confidence threshold.
• oLRP Error finds the minimum LRP Error across all confidence scores.
• Comparative analysis with AP and PQ using numerous state-of-the-art detectors across seven tasks and ten datasets.

Main Results:

• LRP Error provides richer and more discriminative performance insights compared to AP and PQ.
• The proposed metric demonstrates applicability across a wide range of visual detection tasks, including object detection, instance segmentation, and zero-shot detection.
• Empirical evaluation confirms LRP Error's superiority in evaluating visual detectors.

Conclusions:

• LRP Error offers a more robust and informative evaluation metric for visual detection than existing methods.
• The metric's adaptability to various tasks and its ability to assess localization quality make it a valuable tool for researchers and practitioners.
• Optimal LRP Error aids in selecting the best confidence thresholds for real-world deployment.