Inter-rater reliability of functional MRI data quality control assessments: A standardised protocol and practical...

Area of Science:

• Neuroimaging
• Data Science
• Biomedical Engineering

Background:

• Functional magnetic resonance imaging (fMRI) data quality control (QC) is crucial to prevent analysis errors.
• Manual fMRI data inspection is time-consuming and prone to human error.
• Automated tools like pyfMRIqc aim to streamline fMRI QC, but results still require interpretation.

Purpose of the Study:

• To introduce a standardized quality control protocol utilizing pyfMRIqc.
• To evaluate the inter-rater reliability of fMRI data classification using this protocol.
• To identify challenges and propose solutions for consistent fMRI data quality assessment.

Main Methods:

• A protocol for fMRI data quality control using pyfMRIqc was developed.
• Four independent raters classified data from the fMRI Open QC project.
• Data were categorized as 'include,' 'uncertain,' or 'exclude' based on quality.

Main Results:

• Moderate to substantial inter-rater agreement was observed for 'include' and 'exclude' classifications.
• Little to no agreement was found for the 'uncertain' classification.
• The 'uncertain' category was infrequently used by multiple raters for the same dataset.

Conclusions:

• The pyfMRIqc protocol shows promise but requires refinement for the 'uncertain' category.
• Standardizing criteria for uncertain fMRI data can enhance classification consistency.
• Improved inter-rater reliability in fMRI QC is essential for reproducible research.