Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Receiver Operating Characteristic Plot01:15

Receiver Operating Characteristic Plot

570
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
570
Expected Frequencies in Goodness-of-Fit Tests01:19

Expected Frequencies in Goodness-of-Fit Tests

8.8K
A goodness-of-fit test is conducted to determine whether the observed frequency values are statistically similar to the frequencies expected for the dataset. Suppose the expected frequencies for a dataset are equal such as when predicting the frequency of any number appearing when casting a die. In that case, the expected frequency is the ratio of the total number of observations (n)  to the number of categories (k).
8.8K
Dose-Response Relationship: Selectivity and Specificity01:25

Dose-Response Relationship: Selectivity and Specificity

10.5K
Drugs exert their therapeutic effects by interacting with receptors, enzymes, or ion channels that are present throughout the human body. The strength and duration of the interaction between a drug and its target receptor are characterized by the selectivity and specificity of the drug. Selectivity refers to a drug's strong preference for its intended target over other targets. For instance, isoprenaline, a non-selective β-adrenergic agonist, interacts with both β1- and...
10.5K
McNemar's Test01:23

McNemar's Test

974
McNemar's Test is a nonparametric statistical test used to determine if there is a significant difference in proportions between two related groups when the outcome is binary (e.g., yes/no, success/failure). It is beneficial when we have paired data, such as pre-test/post-test designs, where the same subjects are measured under two different conditions. The test is named after the statistician Quinn McNemar, who introduced it in 1947. It is commonly used in situations where subjects are...
974

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Charge Diffusion and Repulsion in Semiconductor Detectors.

Sensors (Basel, Switzerland)·2024
Same author

Quantifying Imaging Agent Binding and Dissociation in 3-D Cancer Spheroid Tissue Culture Using Paired-Agent Principles.

Annals of biomedical engineering·2024
Same author

Improved intraoperative identification of close margins in oral squamous cell carcinoma resections using a dual aperture fluorescence ratio approach: first in-human results.

Journal of biomedical optics·2024
Same author

A cadaveric breast cancer tissue phantom for phase-contrast X-ray imaging applications.

Animal models and experimental medicine·2023
Same author

Myocardial perfusion imaging with retrospective gating and integrated correction of attenuation, scatter, respiration, motion, and arrhythmia.

Journal of nuclear cardiology : official publication of the American Society of Nuclear Cardiology·2023
Same author

Special Section Guest Editorial: Hard X-Ray Tomography with Micrometer Resolution.

Journal of medical imaging (Bellingham, Wash.)·2022

Related Experiment Video

Updated: Mar 26, 2026

A Two-interval Forced-choice Task for Multisensory Comparisons
07:13

A Two-interval Forced-choice Task for Multisensory Comparisons

Published on: November 9, 2018

11.6K

Full receiver operating characteristic curve estimation using two alternative forced choice studies.

Francesc Massanes1, Jovan G Brankov1

  • 1Illinois Institute of Technology , Medical Imaging Research Center, Chicago, Illinois 60616, United States.

Journal of Medical Imaging (Bellingham, Wash.)
|February 12, 2016
PubMed
Summary
This summary is machine-generated.

This study introduces a new method using multiple rounds of two-alternative forced choice (2AFC) studies and the ELO rating system to efficiently estimate medical image quality and generate receiver operating characteristic (ROC) curves, reducing time and cost.

Keywords:
computer tomographyimage evaluationimage qualityreader studies

More Related Videos

Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents
07:05

Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents

Published on: September 10, 2018

6.6K
Testing for Metacognitive Responding Using an Odor-based Delayed Match-to-Sample Test in Rats
08:06

Testing for Metacognitive Responding Using an Odor-based Delayed Match-to-Sample Test in Rats

Published on: June 18, 2018

7.7K

Related Experiment Videos

Last Updated: Mar 26, 2026

A Two-interval Forced-choice Task for Multisensory Comparisons
07:13

A Two-interval Forced-choice Task for Multisensory Comparisons

Published on: November 9, 2018

11.6K
Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents
07:05

Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents

Published on: September 10, 2018

6.6K
Testing for Metacognitive Responding Using an Odor-based Delayed Match-to-Sample Test in Rats
08:06

Testing for Metacognitive Responding Using an Odor-based Delayed Match-to-Sample Test in Rats

Published on: June 18, 2018

7.7K

Area of Science:

  • Medical Imaging
  • Observer Performance Studies
  • Image Quality Assessment

Background:

  • Traditional medical image quality assessment relies on human observer studies, which are time-consuming and costly.
  • Standard psychophysical studies using confidence scoring are resource-intensive and require expert observers.
  • Two-alternative forced choice (2AFC) studies are faster but do not yield full receiver operating characteristic (ROC) curves or individual image scores.

Purpose of the Study:

  • To develop a novel methodology for estimating image confidence scores and generating full ROC curves using multiple rounds of 2AFC studies.
  • To adapt rating systems, such as the ELO rating system, for assessing image quality in a task-based manner.
  • To reduce the time and cost associated with medical image quality evaluation.

Main Methods:

  • Treating image confidence scoring as an unknown rating to be estimated via a two-player match game framework.
  • Employing the ELO rating system (or alternatives like TrueSkill™, Chessmetrics, Glicko) to re-estimate image confidence scores from 2AFC study outcomes.
  • Implementing a pairing strategy starting with abnormal vs. normal images, followed by random pairings in subsequent rounds.

Main Results:

  • Simulated data demonstrated that multiple rounds of 2AFC studies can successfully recover a full ROC curve.
  • The proposed pairing strategy proved effective in recovering diagnostic accuracy metrics.
  • Pilot human observer study results indicated that 3-5 rounds of 2AFC studies are more time-efficient than full scoring studies, with high agreement in ROC curve and area under the ROC curve (AUC) values.

Conclusions:

  • The proposed methodology offers a more efficient approach to medical image quality assessment using 2AFC studies and rating systems.
  • This method significantly reduces human observer time and costs compared to traditional scoring studies.
  • The re-estimated ROC curves and AUC values show high statistical agreement with those from full scoring studies, validating the approach.