Assigning readers to cases in imaging studies using balanced incomplete block designs

Area of Science:

• Medical Imaging Analysis
• Statistical Methodology in Research

Background:

• Imaging studies often involve multiple human readers assessing features.
• Inter-reader agreement is crucial alongside diagnostic accuracy and clinical outcome associations.
• Complete reader designs maximize efficiency but impose a heavy reading burden.

Purpose of the Study:

• To introduce and evaluate balanced incomplete block designs (BIBD) for imaging studies.
• To reduce the significant reading burden associated with complete reader designs.
• To ensure the estimability of inter-reader agreement metrics using BIBD.

Main Methods:

• Utilized balanced incomplete block designs (BIBD) to assign readers to subsets of cases.
• Developed methodologies for data analysis under BIBD.
• Performed power and sample size calculations for BIBD.
• Applied methods to simulation studies and a real-world example.

Main Results:

• BIBD reduced reading burdens by over 40% in simulations.
• Standard errors increased by less than 20% in most scenarios.
• Power to detect differences in diagnostic accuracy decreased by less than 8%.
• Power to detect differences in kappa statistics decreased by less than 20%.

Conclusions:

• Balanced incomplete block designs offer a practical solution to reduce reader burden in imaging studies.
• BIBD effectively maintain the ability to estimate inter-reader agreement.
• The trade-offs in statistical power and precision are generally modest, making BIBD a viable alternative to complete designs.