Related Experiment Video
Updated: Jul 9, 2025
07:37
Assessment and Communication for People with Disorders of Consciousness
Published on: August 1, 2017
9.1K
Confidence score: a data-driven measure for inclusive systematic reviews considering unpublished preprints
Jiayi Tong1,2, Chongliang Luo3, Yifei Sun4
1The Center for Health Analytics and Synthesis of Evidence (CHASE), Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania, Philadelphia, PA 19104, United States.
Journal of the American Medical Informatics Association : JAMIA
|December 8, 2023
Summary
A new confidence score method weights preprints for COVID-19 research. This data-driven approach enhances systematic reviews and meta-analyses by incorporating unpublished manuscripts effectively.
Area of Science:
- Medical Research
- Bibliometrics
- Epidemiology
Background:
- The COVID-19 pandemic spurred a massive increase in research publications, with preprints forming a significant portion.
- The impact and reliability of preprints in evidence synthesis remain unclear.
- Existing methods for evaluating study quality have limitations when assessing preprints.
Purpose of the Study:
- To introduce a novel, data-driven method for assigning weights to preprints in systematic reviews and meta-analyses.
- To develop a "confidence score" that quantifies the reliability of preprints.
- To demonstrate the practical application of this score in COVID-19 research.
Main Methods:
- A survival cure model was employed to calculate the confidence score.
- The model incorporated factors like time to publication, citation counts, sample size, and study type.
- The method was validated using 146 COVID-19 therapeutics preprints.
Main Results:
- The confidence score demonstrated high predictive accuracy with an area under the curve of 0.95 (95% CI, 0.92-0.98).
- A use case involving hydroxychloroquine showed the score's utility in meta-analyses.
- The method provides a supplementary measure to existing quality assessments.
Conclusions:
- The proposed confidence score offers a valuable tool for improving the quality of systematic reviews.
- This data-driven approach facilitates the inclusion of unpublished manuscripts in evidence synthesis.
- The method has broad applicability for COVID-19 and other clinical research areas.
Related Concept Videos
01:24Confidence Coefficient
7.6K
The confidence coefficient is also known as the confidence level or degree of confidence. It is the percent expression for the probability, 1-α, that the confidence interval contains the true population parameter assuming that the confidence interval is obtained after sufficient unbiased sampling; for example, if the CL = 90%, then in 90 out of 100 samples the interval estimate will enclose the true population parameter. Here α is the area under the curve, distributed equally under...
7.6K
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
130
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
130
01:13Sensitivity, Specificity, and Predicted Value
385
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
385
01:21Confidence Intervals
6.3K
An unbiased point estimate is often insufficient to predict a population estimate, such as population mean or population proportion. In this scenario, a confidence interval is used. A confidence interval is an estimate similar to a sample proportion. However, unlike the point estimate which is a single value, the confidence interval contains a range of values. These values have lower and upper limits, known as confidence limits, and can be designated as L1 and L2, respectively.
A...
A...
6.3K
01:16Critical Region, Critical Values and Significance Level
11.9K
The critical region, critical value, and significance level are interdependent concepts crucial in hypothesis testing.
In hypothesis testing, a sample statistic is converted to a test statistic using z, t, or chi-square distribution. A critical region is an area under the curve in probability distributions demarcated by the critical value. When the test statistic falls in this region, it suggests that the null hypothesis must be rejected. As this region contains all those values of the...
In hypothesis testing, a sample statistic is converted to a test statistic using z, t, or chi-square distribution. A critical region is an area under the curve in probability distributions demarcated by the critical value. When the test statistic falls in this region, it suggests that the null hypothesis must be rejected. As this region contains all those values of the...
11.9K
01:29Bias in Epidemiological Studies
287
Biases can arise at various stages of research, from study design and data collection to analysis and interpretation. Recognizing and addressing these biases is essential to ensure the validity and reliability of epidemiological findings.Broadly speaking, biases in epidemiology fall into three main categories: selection bias, information bias, and confounding. A more detailed description of possible biases is:
287