Related Experiment Video
Updated: May 10, 2026
14:05
Behavioral Assessment of Hearing in 2 to 4 Year-old Children: A Two-interval, Observer-based Procedure Using Conditioned Play-based Responses
Published on: January 23, 2017
Choosing thresholds for statistical signal detection with the proportional reporting ratio
Jim Slattery1, Yolanda Alvarez, Ana Hidalgo
1Pharmacovigilance and Risk Management Sector, European Medicines Agency, 7 Westferry Circus, Canary Wharf, London, E14 4HB, UK. Jim.Slattery@ema.europa.eu
Drug Safety
|June 12, 2013
Summary
Adjusting the threshold for drug-event report counts significantly improves adverse drug reaction (ADR) signal detection efficiency. Increasing the threshold reduces false alarms, enhancing the identification of potential drug safety issues.
Area of Science:
- Pharmacovigilance
- Drug Safety
- Regulatory Science
Background:
- Statistical screening algorithms are crucial for identifying drug safety problems from adverse drug reaction (ADR) databases.
- Threshold selection for signal detection parameters directly impacts screening sensitivity and false alarm rates, influencing overall effectiveness.
Purpose of the Study:
- To assess how varying warning thresholds for the proportional reporting ratio (PRR) and report counts affect ADR signal detection.
- To evaluate the trade-offs between sensitivity, timeliness, and false positives in pharmacovigilance signal detection.
Main Methods:
- Utilized the EudraVigilance database to test signal detection methods for suspected ADRs.
- Calculated the number of known ADRs detected and time gained for investigation across a range of thresholds.
- Compared these metrics against the number of false positive signals generated by statistical algorithms.
Main Results:
- Higher PRR thresholds reduced false positives but decreased ADR detection sensitivity.
- Increased thresholds for drug-event report counts yielded fewer false positives with minimal loss in sensitivity or investigation time.
- A report count threshold of five reduced false positives by 25% at the cost of a 12% loss in early true signal detection compared to a threshold of three.
Conclusions:
- Modifying the standard threshold for drug-event report counts can substantially enhance pharmacovigilance signal detection efficiency.
- Initial implementation of threshold changes could be considered for products with established safety profiles.
Related Concept Videos
01:24Testing a Claim about Population Proportion
A complete procedure for testing a claim about a population proportion is provided here.
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
01:15Receiver Operating Characteristic Plot
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
01:09Decision Making: P-value Method
The process of hypothesis testing based on the P-value method includes calculating the P- value using the sample data and interpreting it.
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can have a...
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can have a...
01:14Decision Making: Traditional Method
The process of hypothesis testing based on the traditional method includes calculating the critical value, testing the value of the test statistic using the sample data, and interpreting these values.
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
00:54Ratio Level of Measurement
The way a set of data is measured is called its level of measurement. Correct statistical procedures depend on a researcher being familiar with levels of measurement. For analysis, data are classified into four levels of measurement—nominal, ordinal, interval, and ratio.
A set of data measured using the ratio scale takes care of the ratio problem and provides complete information. Ratio scale data are like interval scale data, except they have a zero point and ratios can be calculated. For...
A set of data measured using the ratio scale takes care of the ratio problem and provides complete information. Ratio scale data are like interval scale data, except they have a zero point and ratios can be calculated. For...
01:16Critical Region, Critical Values and Significance Level
The critical region, critical value, and significance level are interdependent concepts crucial in hypothesis testing.
In hypothesis testing, a sample statistic is converted to a test statistic using z, t, or chi-square distribution. A critical region is an area under the curve in probability distributions demarcated by the critical value. When the test statistic falls in this region, it suggests that the null hypothesis must be rejected. As this region contains all those values of the test...
In hypothesis testing, a sample statistic is converted to a test statistic using z, t, or chi-square distribution. A critical region is an area under the curve in probability distributions demarcated by the critical value. When the test statistic falls in this region, it suggests that the null hypothesis must be rejected. As this region contains all those values of the test...