Cortical thickness in brain imaging studies using FreeSurfer and CAT12: A matter of reproducibility

A reproducibility crisis has been reported across many research fields, including neuroimaging, reaching up to 70% of studies. Neuroimaging data, such as magnetic resonance imaging (MRI), requires pre-processing to allow for inter-subject comparison, increase signal contrast and noise reduction. As manual MRI pre-processing is time consuming and requires expertise, multiple automatic pre-processing frameworks have been proposed. However, neuroimaging studies often report divergent results, even for similar populations, thus it is important to determine whether this occurs as a result of different processing tools. Two of the most used tools are FreeSurfer and the Computational Anatomy Toolbox (CAT12). In this study we assessed the reproducibility between these two automatic pre-processing frameworks for structural MRI and test-retest reliability within framework on estimation of cortical thickness. Our results show that the reproducibility between the frameworks is lower at the region-of-interest (ROI) level than at individual level. Furthermore, we found that the reproducibility was lower in paediatric samples than in adults. Finally, an acquisition site effect was also identified. Given the widespread use of these frameworks in basic and clinical neuroscience, the results of multicentric cross-sectional studies must be interpreted with caution, particularly with paediatric samples. The observed reproducibility issue might be one of the sources of discrepancies reported in neuroimaging studies. On a positive note, framework test-retest reliability within subject is high, suggesting that inconsistency of results may be less concerning in longitudinal studies. The code is available at: https://cibit-uc.github.io/fs-cat12-cortical-thickness-reproducibility.