Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Sample sizes for usability studies: additional considerations

J R Lewis1

  • 1International Business Machines, Inc., Boca Raton, Florida 33429-1328.

Human Factors
|June 1, 1994
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Higher Dietary Vitamin K Intake is Associated with Better Physical Function and Lower Long-Term Injurious Falls Risk in Community-Dwelling Older Women.

The journal of nutrition, health & aging·2023
Same author

Calcaneal quantitative ultrasound is associated with all-cause and cardiovascular disease mortality independent of hip bone mineral density.

Osteoporosis international : a journal established as result of cooperation between the European Foundation for Osteoporosis and the National Osteoporosis Foundation of the USA·2022
Same author

Effects of calcium supplementation on circulating osteocalcin and glycated haemoglobin in older women.

Osteoporosis international : a journal established as result of cooperation between the European Foundation for Osteoporosis and the National Osteoporosis Foundation of the USA·2019
Same author

Establishing an Operational Definition of Sarcopenia in Australia and New Zealand: Delphi Method Based Consensus Statement.

The journal of nutrition, health & aging·2018
Same author

Utility of four sarcopenia criteria for the prediction of falls-related hospitalization in older Australian women.

Osteoporosis international : a journal established as result of cooperation between the European Foundation for Osteoporosis and the National Osteoporosis Foundation of the USA·2018
Same author

Ageing Effects on 3-Dimensional Femoral Neck Cross-Sectional Asymmetry: Implications for Age-Related Bone Fragility in Falling.

Journal of clinical densitometry : the official journal of the International Society for Clinical Densitometry·2018
Same journal

Compatibility Effects With Simple Lever Tools: A Replication and Extension Beyond Simple Button Responses.

Human factors·2026
Same journal

Effects of Egocentric and Exocentric Supervisor Viewpoint Perspectives on Motion Plan Legibility and Decision Support in Automated Spacecraft Docking Maneuvers.

Human factors·2026
Same journal

System-Wide Trust (SWT) Versus Component-Specific Trust (CST) in Multi-Agent Human-Agent Teams: Individual Variability in Trust Bias.

Human factors·2026
Same journal

Driver Adaptation to Partially Automated Driving in Urban Environments: Effects of Repeated Exposure and System Capabilities on Drivers' Trust, Monitoring, and Response.

Human factors·2026
Same journal

Modeling Human Expertise in a Sanding Task.

Human factors·2026
Same journal

Towards Safe and Comfortable Vehicle Control Transitions: A Systematic Review of Takeover Time, Time Budget, and Takeover Outcomes.

Human factors·2026
See all related articles

For usability studies, finding most issues requires fewer participants than previously thought, but severity doesn't guarantee early detection. More participants yield diminishing returns for uncovering usability problems.

Area of Science:

  • Human-Computer Interaction
  • Usability Engineering
  • Product Development

Background:

  • Virzi (1992) proposed that 4-5 participants in usability studies reveal 80% of issues.
  • Subsequent claims suggested diminishing returns and easier detection of severe issues with early participants.

Purpose of the Study:

  • To independently evaluate Virzi's (1992) claims on sample sizes for usability studies.
  • To determine the relationship between participant numbers, problem discovery, and problem severity.

Main Methods:

  • Conducted an independent usability study.
  • Analyzed problem discovery rates against sample size.
  • Correlated problem severity with detection rates.
  • Used economic simulations to assess return on investment (ROI).

Related Experiment Videos

Main Results:

  • Confirmed diminishing returns: more participants uncover fewer new usability problems.
  • Partially supported the 80% problem discovery with 4-5 participants, contingent on a 0.32-0.42 problem detection likelihood.
  • Found no correlation between problem severity and the rate of discovery.
  • Binomial probability formula may model problem discovery curves.
  • Average likelihood of problem detection strongly influenced maximum ROI sample size.

Conclusions:

  • The number of participants needed to find 80% of usability problems depends on the average likelihood of detecting any single problem.
  • Problem severity does not reliably predict how easily issues will be found.
  • Sample size optimization for maximum ROI is primarily driven by the expected problem detection rate.