Measuring the validity and reliability of forensic likelihood-ratio systems
1University of New South Wales, Sydney, Australia.
Science & Justice : Journal of the Forensic Science Society
|September 6, 2011
Summary
Forensic science validity and reliability are crucial. This paper reviews accuracy and precision metrics, like log likelihood-ratio cost (C(llr)) for voice comparison, and introduces improved credible interval calculations.
Area of Science:
- Forensic science
- Forensic voice comparison
Background:
- Growing concerns regarding the validity and reliability of forensic science methods.
- Need for standardized metrics to assess scientific rigor across forensic disciplines.
Purpose of the Study:
- To review metrics for validity and reliability in forensic science, specifically for voice comparison.
- To introduce a revised procedure for calculating credible intervals.
Main Methods:
- Review of existing validity and reliability metrics applied in forensic voice comparison.
- Focus on log likelihood-ratio cost (C(llr)) as a metric for validity.
- Discussion of empirical estimates of credible intervals for reliability.
Main Results:
- The log likelihood-ratio cost (C(llr)) is presented as a key metric for assessing validity.
- An empirical estimation of credible intervals is proposed for reliability assessment.
- The applicability of these metrics to other forensic science branches is highlighted.
Conclusions:
- Established metrics like C(llr) and credible intervals are vital for ensuring scientific validity and reliability in forensic voice comparison.
- The revised procedure for credible intervals offers an improved method for reliability assessment.
- These metrics hold potential for enhancing scientific standards across various forensic disciplines.
Related Concept Videos
01:29Reliability and Validity
14.2K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
14.2K
00:54Ratio Level of Measurement
21.7K
The way a set of data is measured is called its level of measurement. Correct statistical procedures depend on a researcher being familiar with levels of measurement. For analysis, data are classified into four levels of measurement—nominal, ordinal, interval, and ratio.
A set of data measured using the ratio scale takes care of the ratio problem and provides complete information. Ratio scale data are like interval scale data, except they have a zero point and ratios can be calculated....
A set of data measured using the ratio scale takes care of the ratio problem and provides complete information. Ratio scale data are like interval scale data, except they have a zero point and ratios can be calculated....
21.7K
01:09Odds Ratio
1.9K
The odds ratio (OR) is a statistical measure used extensively in epidemiology and research to quantify the strength of association between exposure and outcome across different groups. Unlike relative risk, which compares the probabilities of an event occurring, the odds ratio compares the odds of an event occurring in the exposed group to the odds of it occurring in the unexposed group. The odds, in this context, are calculated as the probability of the event happening divided by the...
1.9K
01:21Friedman Two-way Analysis of Variance by Ranks
514
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
514
01:20Spearman's Rank Correlation Test
1.5K
Spearman's rank correlation test, also known as Spearman's rho, is a nonparametric method for assessing the strength and direction of association between two variables. This test is particularly valuable when the data distribution is unknown or when the assumption of normality does not hold. Named after the English psychologist and statistician Dr. Charles Edward Spearman, it serves as the nonparametric counterpart to Pearson's correlation coefficient.
Spearman's test calculates correlation by...
Spearman's test calculates correlation by...
1.5K
01:17Wald-Wolfowitz Runs Test II
567
The Wald-Wolfowitz runs test, commonly referred to as the runs test, is a nonparametric test used to assess the randomness of ordered data. The test evaluates the number of runs, which are consecutive sequences of similar elements within the data. If the number of runs is significantly higher or lower than expected, the data is considered non-random, indicating a detectable pattern or structure.
For binary data, runs are identified using symbols such as + and −, or equivalently, 1s and 0s. In...
For binary data, runs are identified using symbols such as + and −, or equivalently, 1s and 0s. In...
567