Related Experiment Video
Updated: Oct 3, 2025
Signal Acquisition, Score Interpretation, and Economics of a Non-Invasive Point-of-Care Test for Coronary Artery Disease
Published on: August 9, 2024
Evaluating the Impact of Peer Review on the Completeness of Reporting in Imaging Diagnostic Test Accuracy Research
Sakib Kazi1, Robert A Frank2, Jean-Paul Salameh3,4
1Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada.
Background:
Despite the nearly ubiquitous reported use of peer review among reputable medical journals, there is limited evidence to support the use of peer review to improve the quality of biomedical research and in particular, imaging diagnostic test accuracy (DTA) research.
Purpose:
To evaluate whether peer review of DTA studies published by imaging journals is associated with changes in completeness of reporting, transparency for risk of bias assessment, and spin.
Study Type:
Retrospective cross-sectional study.
Study Sample:
Cross-sectional study of articles published in Journal of Magnetic Resonance Imaging (JMRI), Canadian Association of Radiologists Journal (CARJ), and European Radiology (EuRad) before March 31, 2020.
Assessment:
Initial submitted and final versions of manuscripts were evaluated for completeness of reporting using the Standards for Reporting Diagnostic Accuracy Studies (STARD) 2015 and STARD for Abstracts guidelines, transparency of reporting for risk of bias assessment based on Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2), and actual and potential spin using modified published criteria.
Statistical Tests:
Two-tailed paired t-tests and paired Wilcoxon signed-rank tests were used for comparisons. A P value <0.05 was considered to be statistically significant.
Results:
We included 84 diagnostic accuracy studies accepted by three journals between 2014 and 2020 (JMRI = 30, CARJ = 23, and EuRad = 31) of the 692 which were screened. Completeness of reporting according to STARD 2015 increased significantly between initial submissions and final accepted versions (average reported items: 16.67 vs. 17.47, change of 0.80 [95% confidence interval 0.25-1.17]). No significant difference was found for the reporting of STARD for Abstracts (5.28 vs. 5.25, change of -0.03 [-0.15 to 0.11], P = 0.74), QUADAS-2 (6.08 vs. 6.11, change of 0.03 [-1.00 to 0.50], P = 0.92), actual "spin" (2.36 vs. 2.40, change of 0.04 [0.00 to 1.00], P = 0.39) or potential "spin" (2.93 vs. 2.81, change of -0.12 [-1.00 to 0.00], P = 0.23) practices.
Conclusion:
Peer review is associated with a marginal improvement in completeness of reporting in published imaging DTA studies, but not with improvement in transparency for risk of bias assessment or reduction in spin.
Level Of Evidence:
3 TECHNICAL EFFICACY STAGE: 1.
Insights
Peer review marginally improves reporting completeness in diagnostic test accuracy (DTA) studies but does not enhance transparency or reduce spin. Further research is needed to optimize peer review for DTA research quality.
Area of Science:
- Medical Imaging
- Biomedical Research
- Scientific Publishing
Background:
- Peer review is standard practice in medical journals, yet evidence for its impact on biomedical research quality, especially diagnostic test accuracy (DTA) studies, is limited.
- The effectiveness of peer review in improving the reporting and transparency of DTA research remains uncertain.
Purpose of the Study:
- To assess if peer review influences the completeness of reporting, risk of bias assessment transparency, and "spin" in imaging DTA studies.
- The study aimed to quantify changes between manuscript submission and final publication.
Main Methods:
- A retrospective cross-sectional study analyzed 84 DTA articles from three imaging journals (JMRI, CARJ, EuRad) published before March 31, 2020.
- Manuscripts were evaluated using STARD 2015 and STARD for Abstracts for reporting completeness, QUADAS-2 for bias assessment transparency, and modified criteria for "spin."
Main Results:
- Completeness of reporting based on STARD 2015 showed a statistically significant increase (0.80 items) between initial submission and final versions.
- No significant improvements were observed in STARD for Abstracts reporting, QUADAS-2 transparency, or reduction in actual or potential "spin."
Conclusions:
- Peer review is associated with a minor enhancement in reporting completeness for imaging DTA studies.
- Peer review did not demonstrably improve transparency in risk of bias assessment or decrease "spin" in these studies.
Related Concept Videos
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
01:13Accuracy and Errors in Hypothesis Testing
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
01:15Receiver Operating Characteristic Plot
01:10Computed Tomography
The technique was invented in the 1970s and is based on the principle that as X-rays pass through the body, they are absorbed or reflected at different levels. In the technique, a patient lies on a motorized platform while a computerized axial tomography (CAT) scanner rotates...
01:30Radiological Investigation II: MRI and Ventilation Perfusion Scan
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
01:13Sensitivity, Specificity, and Predicted Value
Sensitivity is the...