A tool for false positive rate estimation in cognitive impairment research: Handling correlated tests, small samples,...

Background: In the field of clinical neuropsychology, researchers and clinicians often use predefined criteria to determine whether there are indications of cognitive impairment. However, corrections and expected false-positive rates are typically available only for uncorrelated tests and simple consensus criteria. Objective: To present an easy-to-use and freely available online tool as a solution for scenarios involving correlated tests and composite consensus criteria, as frequently encountered in clinical neuropsychological research and practice. Method: Our tool employs Monte Carlo simulations to account for the number of participants, thus addressing the uncertainty in estimating false positive rates with small samples. We demonstrate the tool's utility through an example involving cognitive impairment assessment in cancer patients after chemotherapy. Results: The tool reveals considerable uncertainty in false positive rates, especially with small sample sizes, where rates may be higher than traditionally assumed. We found that correlations between tests affect impairment rates differently depending on whether single or multiple test criteria are used. For single-test criteria, lower correlations are associated with more false positives, while for multiple-test criteria, lower correlations lead to fewer false positives. Conclusions: This innovative tool enables more accurate estimation of false positive rates in various neuropsychological conditions. By providing a user-friendly interface and accounting for real-world complexities such as test correlations and composite criteria, our tool empowers clinicians and researchers to:Make informed decisions when interpreting neuropsychological test results.Design more robust research protocols for cognitive impairment studies.Better understand the implications of sample size on false positive rates.