Use and misuse of waterfall plots

Area of Science:

• Oncology
• Radiology
• Clinical Trials

Background:

• Waterfall plots are commonly used to visualize tumor size changes in clinical studies.
• Assessing the criteria for generating waterfall plots and the impact of measurement error is crucial for accurate interpretation.

Purpose of the Study:

• To evaluate the variability in criteria used for generating waterfall plots.
• To assess the impact of interobserver variability and measurement error on waterfall plot generation.

Main Methods:

• Reviewed published waterfall plots to identify criteria variability.
• Compared waterfall plots generated by radiologists and oncologists using CT scans from a phase I solid tumor study.
• Quantified interobserver variability based on Response Evaluation Criteria in Solid Tumors 1.1.

Main Results:

• Substantial variability was found in the criteria used to generate published waterfall plots.
• Statistically significant differences in results were observed between all readers and between oncologists, but not radiologists.
• Observer variability led to differing classifications of patient response (e.g., partial response, stable disease).

Conclusions:

• Waterfall plots exhibit significant variability due to definition criteria and measurement errors.
• Trained radiologists are recommended for generating waterfall plots to ensure consistency.
• Caution is advised when interpreting waterfall plot results in clinical trial contexts.