QRATER: a collaborative and centralized imaging quality control web-based application

Area of Science:

• Neuroscience
• Medical Imaging
• Bioinformatics

Background:

• Manual quality control (QC) is critical in scientific analyses, particularly in neuroscience.
• Existing tools for manual neuroimaging QC lack accessibility, speed, and ease of use.
• A need exists for efficient and collaborative platforms to manage large-scale QC tasks.

Purpose of the Study:

• To introduce Qrater, a containerized web-based Python application for efficient image viewing and rating for QC purposes.
• To assess the performance of Qrater in facilitating manual QC tasks for magnetic resonance (MR) images.
• To evaluate the impact of rater experience on QC task efficiency and agreement using Qrater.

Main Methods:

• Developed Qrater, a web-based Python application for image QC.
• Evaluated Qrater's performance on three MR image QC tasks: raw image QC, linear registration QC, and skull segmentation QC.
• Assessed rater performance metrics including time per image, proportion of failed images, and inter-rater agreement (Fleiss' Kappa, Cohen's Kappa).

Main Results:

• Qrater significantly improved rating speed across all tested QC tasks compared to conventional methods.
• Linear registration and skull segmentation tasks showed particularly substantial time savings using Qrater.
• Inter-rater agreement varied by task and rater experience, with excellent agreement (Cohen's Kappa = 0.83) observed for skull segmentation by experts.

Conclusions:

• Qrater is an effective tool for enhancing the speed and accessibility of manual neuroimaging quality control.
• The application supports collaborative QC efforts, facilitating efficient completion of large datasets.
• Qrater demonstrates utility in improving QC efficiency and agreement in neuroscience research.