Related Experiment Video
Updated: Aug 16, 2026
Erosion Identification in Metacarpophalangeal Joints in Rheumatoid Arthritis using High-Resolution Peripheral Quantitative Computed Tomography
Published on: October 6, 2023
Training and calibration improve inter-reader reliability of joint damage assessment using magnetic resonance image
Paul Bird1, Fredrick Joshua, Marissa Lassere
1Rheumatology Department and the Magnetic Resonance Imaging Department, St. George Hospital, Kogarah, New South Wales, Australia. birdp@sesahs.nsw.gov.au
Objective:
To assess the inter-reader reliability of 3 rheumatologist readers before and after training using 2 methods of assessment: magnetic resonance imaging (MRI) computerized erosion volume assessment and MRI scoring using the OMERACT-5 Rheumatoid Arthritis MRI Score (OM-5 RAMRIS) criteria.
Methods:
Erosion volumes were measured in 10 patients [5 wrist and 5 metacarpophalangeal (MCP) joint studies] with rheumatoid arthritis. Erosion scores were derived from this group and 8 additional subjects to provide a total of 18 subjects (10 wrist and 8 MCP joint studies) with MRI scores for comparison. Subjects were selected from existing MRI databases to provide a spectrum of joint damage for assessment. Initial reading was undertaken after the 2 inexperienced readers were provided with instructions regarding OSIRIS computer software and definitions of the OMERACT score; no other formal training was undertaken. One month after the initial reading, the 2 inexperienced readers undertook a 3 hour training session and all 3 readers then took part in 2 subsequent 2 hour calibration sessions. Each reader then reread the original MRI studies using the computerized erosion volume method and the OMERACT MRI RA score. The interval between the baseline and post-training reading was 2 months. All reading was undertaken on a computer workstation and readers were blinded to other readers' results.
Results:
For the wrist MRI studies, inter-reader agreement improved considerably after training for both the computerized MRI volume method and the OMERACT MRI score [intraclass correlation coefficients (ICC) 0.21 and 0.46, respectively, pre-training; 0.92 and 0.85 post-training]. The correlation between all readers' scores and volumes was excellent at baseline and post-training. For the MCP joint studies, inter-reader agreement was moderate at baseline for the erosion volume and score (ICC 0.51 and 0.61). While there was some improvement in agreement post-training for the scoring method (ICC 0.75), there was no significant improvement for the erosion volumes (ICC 0.58).
Conclusion:
Overall, inter-reader agreement for erosion scoring and volume measurement was higher for the wrist joint. The lack of improvement in the MCP joint region for the erosion volume measurements appears to relate primarily to difficulties in estimating the erosion border in the proximal MCP joints using the manual outlining tool. This limits the usefulness of erosion volume measurements in this joint region.
Insights
Training significantly improved inter-reader reliability for rheumatoid arthritis MRI assessments in wrist joints. However, erosion volume measurement reliability in metacarpophalangeal joints remained limited due to technical challenges.
Area of Science:
- Rheumatology
- Radiology
- Medical Imaging Analysis
Background:
- Rheumatoid arthritis (RA) assessment often relies on imaging to detect joint damage.
- Standardizing interpretation of magnetic resonance imaging (MRI) for RA is crucial for reliable disease monitoring.
Purpose of the Study:
- To evaluate the inter-reader reliability of two MRI assessment methods for rheumatoid arthritis (RA) joint erosion.
- To compare reliability before and after standardized training for rheumatologist readers.
Main Methods:
- Two reader groups assessed erosion volumes and OMERACT-5 Rheumatoid Arthritis MRI Score (OM-5 RAMRIS) in wrist and metacarpophalangeal (MCP) joints.
- Assessments were performed before and after a 3-hour training session and calibration sessions.
- Inter-reader agreement was quantified using intraclass correlation coefficients (ICC).
Main Results:
- For wrist joints, inter-reader agreement significantly improved post-training for both volume (ICC 0.21 to 0.92) and scoring (ICC 0.46 to 0.85) methods.
- Metacarpophalangeal (MCP) joint erosion volume assessment showed limited improvement post-training (ICC 0.51 to 0.58).
- Scoring in MCP joints showed moderate improvement (ICC 0.61 to 0.75).
Conclusions:
- Inter-reader agreement for MRI-based RA joint assessment is higher in wrist joints compared to MCP joints.
- Difficulties in manual outlining of erosion borders in MCP joints limit the reliability of erosion volume measurements.
- Standardized training enhances the reliability of MRI scoring and volume assessment, particularly for wrist joints.
