Inter-rater reliability assessment for the new-born screening quality assurance

Area of Science:

• Clinical Diagnostics
• Laboratory Quality Assurance
• Neonatal Health

Background:

• Newborn screening (NBS) quality is crucial for early detection of genetic disorders.
• Subjective steps in extra-analytical phases, such as dried blood spot (DBS) quality classification and haemoglobin pattern assignment, pose risks to patient safety.
• Inter-rater reliability studies are essential to control and minimize variability in these subjective assessments.

Purpose of the Study:

• To evaluate inter-rater reliability in subjective NBS procedures.
• To assess the effectiveness of implemented measures in improving observer agreement.
• To enhance the overall quality and reliability of NBS results.

Main Methods:

• Dried blood spot (DBS) specimens were utilized for inter-rater studies.
• Ten studies focused on DBS quality classification, and four on haemoglobin pattern assignment.
• Krippendorff's alpha was employed to measure inter-rater reliability, with investigations initiated for alpha values below 0.80.

Main Results:

• Initial inter-rater reliability for both DBS quality classification and haemoglobin pattern assignment was inadequate.
• Key issues identified included undefined criteria for 'scant' DBS and inconsistent reporting of Bart's haemoglobin percentages.
• Enhanced training and consensus on criteria significantly improved alpha estimates, indicating better agreement.

Conclusions:

• Inter-rater reliability assessment effectively identified and controlled subjective activities that introduce variability into NBS.
• The observed improvement in alpha values confirmed the efficacy of the implemented training and consensus-building measures.
• These quality assurance strategies are vital for maintaining the integrity of NBS results and ensuring patient safety.