Reviewing the reviewers: the quality of reporting in three secondary journals

Secondary journals such as ACP Journal Club (ACP), Journal Watch (JW) and Internal Medicine Alert (IMA) have enormous potential to help clinicians remain up to date with medical knowledge. However, for clinicians to evaluate the validity and applicability of new findings, they need information on the study design, methodology and results.

Beginning with the first issue in March 1997, we selected 50 consecutive summaries of studies addressing therapy or prevention and internal medicine content from each of the ACP, JW and IMA. We evaluated the summaries for completeness of reporting key aspects of study design, methodology and results.

All of the summaries in ACP reported study design, as compared with 72% of the summaries in JW and IMA (p < 0.001). In summaries of randomized controlled trials the 3 secondary journals were similar in reporting concealment of patient allocation (none reported this), blinding status of participants (ACP 62%, JW 70% and IMA 70% [p = 0.7]), blinding status of health care providers (ACP 12%, JW 4% and IMA 4% [p = 0.4]) and blinding status of judicial assessors of outcomes (ACP 4%, JW 4% and IMA 0% [p = 0.4]). ACP was the only one to report whether investigators conducted an intention-to-treat analysis (in 38% of summaries [p < 0.001]), and it was more likely than the other 2 journals to report the precision of the treatment effect (as a p value or 95% confidence interval) (ACP 100%, JW 0% and IMA 55% [p < 0.001]).

Although ACP provided more information on study design, methodology and results, all 3 secondary journals often omitted important information. More complete reporting is necessary for secondary journals to fulfill their potential to help clinicians evaluate the medical literature.