Related Experiment Videos
Intra- and inter-observer agreement of brain MRI lesion volume measurements in multiple sclerosis. A comparison of
M Filippi1, M A Horsfield, S Bressi
1Department of Neurology, Scientific Institute Ospedale San Raffaele, University of Milan, Italy.
Abstract:
The measurement of MRI lesion load in multiple sclerosis is increasingly being used to evaluate the natural history of the disease and to monitor the efficacy of treatments. If, as might occur in multicentre studies, lesion load is measured by several observers in different patients or by the same observer in serial scans, it would be necessary to utilize a technique which provides results with high inter- and intra-observer agreements. This study was performed to evaluate the intra- and inter-observer agreement of semi-automated lesion volume measurement using thresholding, and to compare them with those obtained using an arbitrary scoring system (ASS) and a quantitative manual tracing method (MTM). Brain MRIs were obtained for 20 clinically definite multiple sclerosis patients and were evaluated independently by three observers. The median intra- and inter-observer agreements were, respectively, 88.5% (range 69.0-96.8%) and 79.0% (range 73.3-98.3%) using the ASS, 95.0% (range 85.1-99.4%) and 93.4% (range 77.3-98.3%) for the MTM, 96.3% (range 94.2-98.9%) and 93.7% (range 83.8-98.3%) for the semi-automated technique. The intra- and inter-observer agreements for the semi-automated technique increased to 98.5% (range 96.3-99.8%) and 96.1% (range 90.5-98.6%) when a consensus in the choice of threshold for lesion segmentation was reached. The intra- and inter-observer agreements were significantly greater for the semi-automated method compared with both the arbitrary scoring and the MTMs. The intra-observer variability for the semi-automated technique was significantly lower (P < 0.0001) than the inter-observer variability obtained using the same technique. These data indicate that it is possible to obtain high intra- and inter-observer agreements using a semi-automatic thresholding technique to quantify lesion volumes in multiple sclerosis. The technique may prove useful in multicentre studies, in which a single observer is still preferable.
Insights
A new semi-automated MRI technique significantly improves agreement for measuring multiple sclerosis lesion load. This method offers higher accuracy than manual or arbitrary scoring, crucial for multicenter studies and treatment monitoring.
Area of Science:
- Neurology
- Medical Imaging
- Biostatistics
Background:
- Magnetic Resonance Imaging (MRI) lesion load is vital for tracking multiple sclerosis (MS) progression and treatment effectiveness.
- Accurate and reproducible lesion measurement is essential, especially in multicenter studies involving multiple observers or serial scans.
- Current methods like arbitrary scoring systems (ASS) and manual tracing methods (MTM) may lack sufficient inter- and intra-observer agreement.
Purpose of the Study:
- To evaluate the intra- and inter-observer agreement of a semi-automated lesion volume measurement technique using thresholding in multiple sclerosis (MS).
- To compare the agreement of the semi-automated technique with ASS and MTM.
- To assess the reliability of the semi-automated technique for quantifying MS lesion volumes.
Main Methods:
- Brain MRIs from 20 patients with clinically definite multiple sclerosis were independently analyzed by three observers.
- Lesion volumes were measured using an arbitrary scoring system (ASS), a manual tracing method (MTM), and a semi-automated thresholding technique.
- Intra- and inter-observer agreements were calculated for each method, with a consensus threshold evaluation for the semi-automated technique.
Main Results:
- The semi-automated technique demonstrated higher median intra-observer agreement (96.3%) and inter-observer agreement (93.7%) compared to ASS and MTM.
- When a consensus threshold was used, the semi-automated technique achieved even higher agreements: 98.5% (intra-observer) and 96.1% (inter-observer).
- The semi-automated method showed significantly greater agreement than both ASS and MTM, with significantly lower intra-observer variability (P < 0.0001).
Conclusions:
- Semi-automated thresholding is a reliable technique for quantifying lesion volumes in multiple sclerosis (MS) with high intra- and inter-observer agreements.
- This technique offers superior reproducibility compared to arbitrary scoring and manual tracing methods.
- The semi-automated approach is well-suited for multicenter studies, though a single observer is still recommended for optimal results.