Interpretable Bayesian Modeling for Multireader Multicase Studies: Addressing Overdispersion and Limited Sample Size...

The advent of machine and deep learning in the medical domain has led to significant advancements in diagnostic workflows and clinical decision-making, making rigorous evaluation of novel techniques essential for their integration into clinical practice. In this work, we introduce a Bayesian hierarchical Beta-Binomial modeling framework for estimating the effect of novel techniques in binary classification, with a particular focus on multireader, multicase study designs, motivated by applications in medical imaging. Some challenges in this context include, small sample sizes (i.e., only few readers particicipating in the study), pronounced overdispersion due to heterogeneity in reader performance, and class-imbalanced datasets. Addiotionally, the actual effect size of the novel technique may be small, further complicating robust estimation of model parameters. The proposed model explicitly accounts for overdispersion, addresses class imbalance within the test cohort, and incorporates prior information to regularize population-level parameter estimates across readers. Through simulation studies, the approach demonstrates improved robustness and lower estimation error compared to classical linear models, especially under high overdispersion and low sample sizes. Application to a real-world study of chest X-ray imaging with and without Bone Suppression Imaging enhancement illustrates the model's practical utility and highlights the importance of accounting for overdispersion and prior information in study design and analysis.