Can We Trust PAICs in Rare Diseases? Methodological Challenges and Limitations

Area of Science:

• Health Technology Assessment
• Comparative Effectiveness Research
• Biostatistics

Background:

• Population-adjusted indirect comparisons (PAICs) are increasingly utilized for health technology assessments (HTA) to generate comparative evidence when head-to-head trials are absent.
• Standard indirect treatment comparison methods are often unfeasible, making PAICs, including Matching-Adjusted Indirect Comparison and Simulated Treatment Comparison, a pragmatic alternative.
• Rare diseases pose unique challenges for PAICs due to small sample sizes, limited covariate overlap, and reliance on unanchored comparisons with unverifiable assumptions.

Purpose of the Study:

• To critically evaluate the methodological challenges and limitations of PAICs, particularly in the context of rare diseases.
• To discuss the implications of these limitations on the reliability and interpretation of PAIC-derived evidence in HTA.
• To propose a nuanced view on the role of PAICs within a broader HTA framework and suggest future research directions.

Main Methods:

• Review and analysis of existing literature on PAICs and their application in HTA, with a focus on rare disease contexts.
• Discussion of methodological refinements such as optimized weighting, Bayesian approaches, and doubly robust estimators.
• Consideration of current regulatory guidance, such as European Joint Clinical Assessment recommendations.

Main Results:

• PAICs in rare diseases are susceptible to substantial methodological challenges, including unstable estimates, reduced precision, and potential bias, stemming from small sample sizes and limited covariate overlap.
• Methodological refinements offer partial improvements but do not fully resolve the fundamental issues associated with PAICs, especially unanchored comparisons.
• European guidance advises extreme caution with anchored PAICs and considers unanchored PAICs highly problematic, recommending alternative methods.

Conclusions:

• PAICs can serve a supportive role in HTA for rare diseases, complementing other evidence sources when data are scarce, but require careful interpretation and transparent communication of uncertainty.
• The reliability of PAIC findings in rare diseases is often undermined by inherent methodological limitations, necessitating a cautious approach.
• Future research must focus on developing formal frameworks to quantify bias and assess robustness to prevent overstating the credibility of PAIC-derived evidence in rare disease settings.