The PRECIS-2 tool has good interrater reliability and modest discriminant validity

Area of Science:

• Clinical trial methodology
• Health services research

Background:

• The PRagmatic Explanatory Continuum Indicator Summary (PRECIS)-2 is a tool designed to enhance understanding of clinical trial designs.
• Previous versions have aimed to guide trialists, but validation of its psychometric properties is crucial.

Purpose of the Study:

• To validate the PRECIS-2 tool by assessing its discriminant validity and interrater reliability.
• To confirm its utility in distinguishing between pragmatic and explanatory trial approaches.

Main Methods:

• PRECIS-2 was developed with input from over 80 international experts to ensure face and content validity.
• Interrater reliability was assessed using 19 experienced trialists scoring 15 randomized controlled trial protocols.
• Discriminant validity was tested by evaluating the tool's ability to differentiate between pragmatic and explanatory trial designs.

Main Results:

• Generally good interrater reliability was observed, with seven of nine domains showing an intraclass correlation coefficient above 0.65.
• While flexibility (adherence) and recruitment domains had wider confidence intervals, all nine domains demonstrated better-than-chance discrimination between pragmatic and explanatory trial approaches.
• Raters identified a need for more information on specific domains like flexibility and recruitment.

Conclusions:

• The PRECIS-2 tool demonstrates acceptable validity and reliability for assessing clinical trial design.
• Further guidance and an accompanying website are available to support users.
• Ongoing testing by trial teams, systematic reviewers, and funders will continue to refine its application.