The variability of manual and computer assisted quantification of multiple sclerosis lesion volumes

The high resolution and excellent soft tissue contrast of Magnetic Resonance Imaging (MRI) have enabled direct, noninvasive visualization of Multiple Sclerosis (MS) lesions in vivo. This has allowed the quantification of changes in the appearance of lesions in MR exams to be used as a measure of disease state. Nevertheless, accurate quantification techniques are subject to inter- and intra-operator variability, which may hinder monitoring of disease progression. We have developed a computer program to assist an experienced operator in the quantification of MS lesions in standard spin-echo MR exams. The accuracy of assisted and manual quantification under known conditions was studied using exams of a test phantom, while inter- and intra-operator reliability and variability were studied using exams of a MS patient. Results from the phantom study show that accuracy is improved by assisted quantification. The patient exam results indicate that assisted quantification reduced inter-operator variability from 0.34 to 0.17 cm3, and reduced intra-operator variability from 0.23 to 0.15 cm3. In addition, the minimum significant change between two successive measurements of lesion volume by the same operator was 0.64 cm3 for manual quantification and 0.42 cm3 for assisted quantification. For two different operators making successive measurements, the minimum significant change was 0.94 cm3 for manual quantification, but only 0.47 cm3 for assisted quantification. Finally, the number of lesions to be monitored for an average change in volume at a given power and significance level was reduced by a factor of 2-4 by assisted quantification. These results suggest that assisted quantification may have practical applications in clinical trials, especially those that are large, multicenter, or extended over time, and therefore require lesion measurements by one or more operators.