Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Search research articles

Related Experiment Videos

Global and partial agreement among several observers

A Rogel¹, P Y Boëlle, J Y Mary

¹Centre de Bioinformatique, INSERM U444, Université Paris 7, France.

Statistics in Medicine

|March 13, 1998

Summary

This study introduces a new global and partial agreement model for analyzing ratings from multiple observers, outperforming traditional pairwise methods, especially with varied expert opinions.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Shift in emergency department utilization by frequent attendees with sickle cell disease during the COVID-19 pandemic: A multicentre cohort study.

British journal of haematology·2024

Same author

Defining and Assessing the Reproducibility of Crohn's Disease Endoscopic Lesions: A Delphi-like Method from the GETAID.

Journal of Crohn's & colitis·2020

Same author

Performances of statistical methods for the detection of seasonal influenza epidemics using a consensus-based gold standard.

Epidemiology and infection·2017

Same author

Defining endoscopic response and remission in ulcerative colitis clinical trials: an international consensus.

Alimentary pharmacology & therapeutics·2017

Same author

Long-term outcome of patients with steroid-refractory acute severe UC treated with ciclosporin or infliximab.

Gut·2017

Same author

IOIBD technical review on endoscopic indices for Crohn's disease clinical trials.

Gut·2015

Area of Science:

Statistics
Medical Diagnostics
Pathology

Background:

Established methods for observer agreement analysis often rely on pairwise comparisons.
Analyzing agreement among more than two observers presents unique statistical challenges.
Existing methods may not fully capture complex interactions in multi-observer datasets.

Purpose of the Study:

To present a novel global and partial agreement modelling approach for multi-observer categorical data.
To address limitations of pairwise agreement analysis in scenarios with multiple raters.
To apply and evaluate this new approach in a real-world diagnostic setting.

Main Methods:

Developed a global and partial agreement modelling approach using quasi-independence and quasi-symmetry log-linear models.
Focused on addressing higher-order interactions within contingency tables.
Applied pairwise, global, and partial agreement models to a dataset of pathologist ratings.

Main Results:

The global and partial agreement modelling approach was applied to detect three lesions by six pathologists in Crohn's disease patient biopsies.
Demonstrated that the global and partial agreement approach is superior to pairwise methods.
Highlighted the enhanced performance of the new approach when rating heterogeneity is present.

Conclusions:

The proposed global and partial agreement modelling offers a more comprehensive analysis of multi-observer agreement.
This method effectively captures higher-order interactions, providing deeper insights than pairwise comparisons.
The approach shows significant advantages in diagnostic settings with multiple expert raters, particularly in complex cases like Crohn's disease detection.

Related Experiment Videos