Model Quality in AI-based Bruise Detection: Rethinking IoU and Confidence Thresholds

Area of Science:

• Computer Science
• Artificial Intelligence
• Medical Imaging

Background:

• AI object detection models (e.g., YOLO, Faster R-CNN) rely on Intersection over Union (IoU) and confidence thresholds for performance evaluation.
• Fixed thresholds can introduce bias and disparate impacts across subpopulations, masking underlying fairness issues despite strong overall performance metrics.

Purpose of the Study:

• To investigate the impact of varying IoU and confidence thresholds on standard performance metrics (precision, recall, F1-score, mAP) and fairness metrics (Demographic Parity, Equalized Odds, Equality of Opportunity, Accuracy Equality, Disparate Impact).
• To explore fairness and performance trade-offs in AI-driven bruise detection under different lighting conditions.

Main Methods:

• Evaluated AI object detection models using a dataset of bruise images captured under natural and alternative light sources.
• Analyzed the effects of systematically varying IoU and confidence thresholds on multiple performance and fairness metrics.

Main Results:

• Fairness and performance trade-offs were observed to be mitigated by selecting intermediate threshold values over fixed extreme values.
• The study demonstrated that specific threshold choices significantly influence model fairness and performance outcomes.

Conclusions:

• Dynamical optimization of thresholds is necessary to achieve both high model performance and fairness in AI-driven bruise detection.
• Intermediate threshold values offer a promising strategy for mitigating bias in AI object detection systems for medical applications.