How to assess intra- and inter-observer agreement with quantitative PET using variance component analysis: a proposal...

Area of Science:

• Medical Imaging
• Quantitative Analysis
• Radiopharmaceutical Science

Background:

• Accurate and precise quantitative measurement procedures are crucial for clinical validation.
• Precision in measurements can be expressed using various metrics like agreement proportions, standard errors, coefficients of variation, or Bland-Altman plots.
• Variance Component Analysis (VCA) is proposed to estimate errors from individual PET scan elements (scanner, time, observer) and quantify composite uncertainty of repeated measurements.

Purpose of the Study:

• To present and evaluate Variance Component Analysis (VCA) for assessing intra- and inter-observer variation in PET/CT scans.
• To demonstrate the application of VCA using data from two clinical studies.
• To establish Repeatability Coefficients (RCs) derived from VCA and their relation to Bland-Altman plots.

Main Methods:

• Study 1 involved 30 ovarian cancer patients scanned pre-operatively, with assessments performed twice by the same observer to evaluate intra-observer agreement using SUVmax.
• Study 2 included 14 glioma patients scanned up to five times (49 scans total), with assessments by three observers to examine inter-observer agreement using cerebral total hemispheric glycolysis (THG).
• VCA, utilizing linear mixed effects models, was employed to estimate variance components and calculate RCs.

Main Results:

• In Study 1, a Repeatability Coefficient (RC) of 2.46 was found, equivalent to half the width of the Bland-Altman limits of agreement.
• In Study 2, the RC under identical conditions (same scanner, patient, time, observer) was 2392, increasing to 2543 when different scanners were included.
• Inter-observer differences were found to be negligible compared to other sources of variation.

Conclusions:

• Variance Component Analysis (VCA) offers an effective method for evaluating and comparing different sources of variation in PET/CT measurements, summarized by RCs.
• The use of linear mixed effects models in VCA necessitates careful consideration of sample sizes to ensure accurate estimation of variance components.
• VCA provides a robust framework for enhancing the precision and reliability of quantitative PET/CT imaging in clinical settings.