Reliable support: Measuring calibration of likelihood ratios

Area of Science:

• Forensic Science
• Statistics
• Evidence Evaluation

Background:

• Calculating likelihood ratios (LR) in forensic evidence evaluation faces challenges.
• Factors like database selection, statistical models, and evidence quality can lead to inaccurate LR values, potentially supporting incorrect propositions.
• Measuring the performance of LR values is not straightforward, necessitating defined metrics.

Purpose of the Study:

• To introduce and describe the concept of calibration as a desirable property of LR values.
• To propose a tool, the Empirical Cross-Entropy (ECE) plot, for representing and measuring the performance and calibration of LR values.
• To demonstrate the utility of ECE plots and calibration measurement using speech evidence examples.

Main Methods:

• Conceptualization of calibration as a key property for LR values.
• Development and application of the Empirical Cross-Entropy (ECE) plot for performance visualization.
• Empirical testing and demonstration using speech evidence data.

Main Results:

• Well-calibrated LR values exhibit desirable performance characteristics in evidence evaluation.
• The ECE plot effectively measures the calibration of LR values.
• The study demonstrates the practical usefulness of ECE plots for assessing LR performance in forensic applications.

Conclusions:

• Calibration is a crucial property for reliable likelihood ratios in forensic science.
• ECE plots provide a valuable tool for quantifying and visualizing LR calibration and performance.
• The proposed methods enhance the rigor of evidence evaluation in forensic disciplines.