When and how to replace a new reference standard for medical tests: a scoping review identifying methodological...

The reference standard, defined as the best available test or test strategy for diagnosing a target disease/condition in a specific clinical population, is rarely perfectly accurate. This imperfection raises the question of when (and how) to establish a new test as the reference standard test. The objective of this scoping review was to assess methodological priorities for establishing a new reference standard for medical tests.

This scoping review was performed using JBI methodology and reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for scoping reviews reporting guideline. We included sources describing methodological approaches to establishing a new reference standard across all healthcare populations and settings. Data were analyzed using a combination of descriptive and qualitative content and thematic analyses.

We screened 5396 studies and included 13 in our final analysis (7 methodological studies and 6 applied studies). Three main methodological themes emerged: (1) demonstrating the imperfectness of the current reference standard, (2) comparing candidate reference standard tests, and (3) decision principles for accepting a new reference standard. Among the six applied studies that established reference standards for specific conditions, five used composite reference standards.

Based on the findings of our review, we define a potential workflow for structuring decision-making relating to whether to accept a new test as the reference standard. Development of formal methodological guidance for when and how to establish or replace a reference standard would likely benefit guideline developers, clinicians, technology assessors, and, ultimately, patients.

"Reference standard" tests are considered the best available method to determine whether someone has a particular health condition. However, these tests are often imperfect, and there is little guidance on when a new test should replace the current reference standard. We reviewed published research to understand how new reference standards have been created or evaluated. Thirteen studies met our search criteria. Together, they showed four key steps: first, defining the purpose of the reference standard (for clinical care or research); second recognizing when the current reference standard is imperfect; third, comparing potential new tests; and fourth, deciding whether adopting a new test will improve patient care. Our findings highlight the need for formal guidance to support groups creating new reference standards in the future.