Related Experiment Video
Updated: Jan 3, 2026

Author Spotlight: Assessing the Reliability of Doppler Ultrasound in Measuring Leg Blood Flow
Published on: December 15, 2023
Intra-observer and Device-Dependent Inter-observer Reliability of Contrast-Enhanced Ultrasound for Muscle Perfusion
Pierre Kunz1, Sophia Kiesl2, Sascha Groß2
1Center for Orthopedics, Trauma Surgery and Spinal Cord Injury, Ultrasound Centre, HTRG, Heidelberg University Hospital, Heidelberg, Germany; Clinic for Shoulder and Elbow Surgery, Catholic Hospital Mainz, Mainz, Germany.
This study assesses how consistently different doctors can measure blood flow in shoulder muscles using ultrasound with contrast agents. The researchers found that while measurements are very reliable when performed by the same person, using different ultrasound machines can affect the results. However, ensuring that the exact same muscle slice is imaged significantly improves consistency across different devices and observers.
Area of Science:
- Diagnostic imaging and Contrast-Enhanced Ultrasound reliability research
- Musculoskeletal medicine and clinical physiology
Background:
No prior work had resolved the consistency of blood flow measurements in shoulder muscles using specialized imaging. That uncertainty drove the need to evaluate if these techniques remain stable across different clinical settings. It was already known that such assessments could guide medical choices for various joint conditions. However, the field lacked data on how different machines influence these specific diagnostic outcomes. Prior research has shown that standardized protocols are often required for multi-site investigations. This gap motivated a closer look at the reproducibility of these diagnostic procedures. Researchers needed to determine if human error or hardware variations primarily impact the quality of the data. Establishing these benchmarks is necessary for the widespread adoption of these imaging tools in daily practice.
Purpose Of The Study:
The aim of this investigation is to determine the reliability of blood flow quantification in the deltoid muscle using specialized imaging. This study addresses the need for consistent diagnostic metrics to support multicenter clinical trials. The researchers seek to identify how observer variability and hardware differences impact the precision of these measurements. No prior work had resolved the specific influence of ultrasound device transferability on perfusion data. That uncertainty drove the team to examine if these techniques can reliably guide treatment decisions in musculoskeletal care. The investigators evaluate whether standardized imaging planes can mitigate the inconsistencies observed during routine clinical practice. Establishing these benchmarks is necessary for the broader application of this technology in diverse medical settings. The project provides a foundation for future protocols aimed at improving the reproducibility of non-invasive muscle blood flow assessments.
Main Methods:
The team conducted a retrospective analysis of 166 imaging sequences obtained from 42 distinct shoulder examinations. Review approach involved calculating the intra-class correlation coefficient to determine the strength of agreement between different raters. The investigators also employed the coefficient of variation to assess the spread of the recorded perfusion values. They stratified the data into subgroups based on the accuracy of the anatomical sectional plane. This design allowed the researchers to isolate the impact of hardware differences from human measurement errors. The study compared results from identical devices against those generated by heterogeneous equipment sets. Statistical evaluation focused on identifying thresholds for acceptable reproducibility in a clinical environment. This systematic process provided a clear view of how technical variables influence the stability of the diagnostic output.
Main Results:
The strongest finding shows that intra-observer reliability reached an intra-class correlation coefficient of 0.91 with a coefficient of variation of 10.28%. Inter-observer reliability initially yielded an intra-class correlation coefficient of 0.84 and a coefficient of variation of 17.1%. Using different ultrasound devices caused these reliability metrics to decline to an intra-class correlation coefficient of 0.60 and a coefficient of variation of 18.6%. Subgroup analysis revealed that high sectional plane concordance improved intra-observer reliability to an intra-class correlation coefficient of 0.97. In this same high-concordance group, the coefficient of variation for intra-observer measurements dropped to 5.49%. Inter-observer reliability for identical devices rose to an intra-class correlation coefficient of 0.98 with a coefficient of variation of 5.83%. When using varying devices with high concordance, the intra-class correlation coefficient reached 0.78 with a coefficient of variation of 9.8%. These figures demonstrate that anatomical alignment is a key driver of measurement stability across different testing conditions.
Conclusions:
The authors suggest that measuring blood flow in the deltoid muscle is feasible for large-scale collaborative research efforts. They propose that pooling data from varying hardware platforms requires careful consideration to maintain accuracy. The team notes that alignment of the imaging plane significantly boosts the consistency of the findings. Their data indicate that high agreement between observers is achievable under specific standardized conditions. The researchers conclude that hardware differences represent a notable challenge for cross-site data integration. They emphasize that precise anatomical positioning remains a primary factor for reliable diagnostic performance. The study implies that future protocols must prioritize consistent sectional views to ensure valid comparisons. These findings provide a framework for integrating advanced imaging into broader clinical diagnostic pathways.
Frequently Asked Questions
The researchers report an intra-class correlation coefficient of 0.91 for intra-observer reliability. In contrast, inter-observer reliability across different ultrasound machines yielded a lower coefficient of 0.60, indicating that hardware variation influences the consistency of the perfusion metrics significantly.
The study utilizes Contrast-Enhanced Ultrasound, a diagnostic imaging technique involving microbubble agents. This approach allows for the quantification of blood flow dynamics within the deltoid muscle, which serves as the specific anatomical target for assessing measurement reproducibility.
The authors propose that high sectional plane concordance is necessary to achieve optimal reliability. This alignment ensures that the same anatomical region is analyzed, which mitigates the variability observed when different clinicians or hardware systems are employed.
The researchers use the intra-class correlation coefficient and the coefficient of variation to quantify data consistency. These statistical metrics allow the team to compare the performance of individual observers against the variability introduced by different ultrasound hardware platforms.
The team measured blood flow in the deltoid muscle across 42 shoulders. They observed that when the imaging plane matched perfectly, the coefficient of variation dropped to 5.49% for intra-observer tests, compared to 10.28% in the initial broader analysis.
The authors suggest that while this imaging method is suitable for multicenter trials, investigators must exercise caution when combining data from diverse ultrasound devices. They propose that standardizing the anatomical view is a prerequisite for reliable cross-site data pooling.

