Related Experiment Videos
Illusions of objectivity and a recommendation for reporting data mining results
Manfred Hauben1, Lester Reich, Charles M Gerrits
1Department of Medicine, New York University School of Medicine, Valhalla, NY, USA.
European Journal of Clinical Pharmacology
|March 17, 2007
Summary
Comparing two data mining algorithms (DMAs) on drug-event combinations revealed significant differences in signal detection. Vendor choice and configuration impact results, questioning the objectivity of DMA findings in pharmacovigilance.
Area of Science:
- Pharmacovigilance
- Data Mining
- Drug Safety
Background:
- Spontaneous reporting system (SRS) databases are crucial for post-licensure drug safety surveillance.
- Data mining algorithms (DMAs) are increasingly used to analyze SRS data for timely safety insights.
- The
- objectivity
- of DMAs is questioned due to numerous configurable parameters and vendor-specific differences.
Purpose of the Study:
- To compare the results of two commonly used DMAs applied to pre-selected drug-event combinations (DECs).
- To assess the impact of vendor choice and configuration on DMA outcomes in pharmacovigilance.
Main Methods:
- Two DMAs were retrospectively applied to the USFDA safety database (up to Q2 2005).
- Eight pre-selected DECs were analyzed using three different thresholds.
- Similar protocols were employed to ensure comparability between software programs.
Main Results:
- Significant differences were observed between the two vendors in signal of disproportionate reporting (SDR) metrics.
- Variations included the number of cases with SDR, the first year of SDRs, and SDR score magnitudes.
- These differences were potentially significant for 45.8% of the analyzed data points.
Conclusions:
- Vendor-specific data cleaning, transformation methods, and algorithm features contribute to observed discrepancies.
- While configurable, DMA choices raise concerns about the claimed objectivity of findings and susceptibility to confirmation bias.
- Transparency is essential: reporting DMA results requires specifying the vendor and all configuration details.
Related Concept Videos
01:25Types of Reports I: Hand-off Report
A hand-off report, also known as a change-of-shift report, is a crucial nursing process that ensures the smooth transition of patient care responsibilities between nursing staff.
Following are the key components and categories of hand-off reports:
Purpose and Process:
Following are the key components and categories of hand-off reports:
Purpose and Process:
01:24Data Reporting and Recording
Reporting and recording are crucial in data documentation. The timely, thorough, and accurate documentation of facts is essential when recording patient data. Failure to record findings during an assessment or interpretation of a problem will result in loss of information and make the patient document unreliable. The reader is left with general impressions if the information is not specific. A recording is documenting data of the individual's health information in a traceable, secure, and...
01:11Statistical Analysis: Overview
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
01:22Bias
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
01:56Ethics in Research
Today, scientists agree that good research is ethical in nature and is guided by a basic respect for human dignity and safety. However, this has not always been the case. Modern researchers must demonstrate that the research they perform is ethically sound.
01:16Statistical Inference Techniques in Hypothesis Testing: Parametric Versus Nonparametric Data
Statistical inference techniques, paramount in hypothesis testing, differentiate into two broad categories: parametric and nonparametric statistics.
Parametric statistics, as the name suggests, assumes that data follow a specific distribution, often a normal distribution. This assumption enables robust hypothesis testing and estimation. Parametric methods, like the Student's t-test or Goodness-of-fit test, are frequently employed in biostatistics due to their robustness. For instance, comparing...
Parametric statistics, as the name suggests, assumes that data follow a specific distribution, often a normal distribution. This assumption enables robust hypothesis testing and estimation. Parametric methods, like the Student's t-test or Goodness-of-fit test, are frequently employed in biostatistics due to their robustness. For instance, comparing...