Common methodological pitfalls in ICI pneumonitis risk prediction studies

Area of Science:

• Oncology
• Immunology
• Pulmonology

Background:

• Pneumonitis is a frequent and potentially fatal adverse event associated with immune checkpoint inhibitors (ICIs).
• Current risk prediction models for ICI-induced pneumonitis are not clinically deployed, hindering patient risk stratification and monitoring.
• Existing literature suffers from suboptimal study designs and methodologies, preventing effective risk assessment.

Purpose of the Study:

• To systematically review and analyze the methodological approaches used in studies identifying risk factors and developing predictive models for ICI-induced pneumonitis.
• To identify common methodological pitfalls and assess their contribution to the risk of bias in existing research.
• To highlight the need for high-quality studies to develop clinically deployable risk prediction models.

Main Methods:

• A systematic literature search was conducted on PubMed, medRxiv, and bioRxiv for studies on ICI-induced pneumonitis risk factors and predictive models.
• Eligible studies were analyzed for common methodological pitfalls using QUIPS and PROBAST tools to assess risk of bias.
• Methodological practices, diagnostic methods, statistical analyses, and handling of missing data were critically examined.

Main Results:

• Out of 51 eligible manuscripts, only 2 demonstrated an overall low risk of bias, indicating widespread methodological limitations.
• Common bias-inducing practices included unclear diagnostic criteria, lack of multiple testing correction, inappropriate feature selection, variable discretization, and poor handling of missing data.
• Risk models were likely overoptimistic due to a lack of holdout sets, suggesting unreliable predictions.

Conclusions:

• The current body of research on ICI-induced pneumonitis is characterized by a high risk of bias, limiting the development of reliable predictive models.
• There is an urgent need for methodologically rigorous studies focusing on accurate risk factor identification and robust model development.
• Recommendations and alternative approaches are discussed to guide future research towards creating clinically useful risk prediction tools for ICI pneumonitis.