Related Experiment Video
Updated: Aug 4, 2026

Measuring Attention and Visual Processing Speed by Model-based Analysis of Temporal-order Judgments
Published on: January 23, 2017
Multipoint rank-order study methodology: observer issues
J D Towers1, J M Holbert, C A Britton
1Department of Radiology, University of Pittsburgh, Pennsylvania 15261-0001, USA.
Rationale And Objectives:
We performed a multipoint rank-order experiment to evaluate variability in observers' sensitivity to small differences in image presentation and to assess observers' performance as a function of the type and number of tasks included.
Methods:
Five experienced observers were presented with four sets of chest images that had been compressed at five different levels. Each set contained six images ranging from noncompressed to approximately 60:1-compressed images. Observers were asked to review all images of each case side by side and rank-order the "quality" of each to enable determination of the presence or absence of interstitial disease and/or pneumothoraces.
Results:
Observers varied significantly in their ability to detect very small differences among the images (P < 0.001). Those who performed well did so regardless of whether they ranked a specific abnormality in a multidisease or a single-disease setting.
Conclusions:
Selected observers can reliably detect very small differences among similar images. These readers could be used to confirm or rule out the need for objective observer-performance-type studies.
Related Concept Videos
Naturalistic Observations
Ordinal Level of Measurement
Data measured using an ordinal scale are similar to nominal scale data, but there is one major difference. The ordinal scale data can be ordered. An example of ordinal scale data is a list of the top five national parks in the...
Observational Studies
There are three types of observational studies – Prospective, retrospective, and cross-sectional.
Prospective Study
Prospective studies, also known as longitudinal or cohort studies, are carried out by collecting future data from groups sharing similar characteristics. One example of...
Ranks
Statistical Analysis: Overview
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
Friedman Two-way Analysis of Variance by Ranks

