Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Multiple Comparison Tests01:13

Multiple Comparison Tests

Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving01:29

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
Comparing the Survival Analysis of Two or More Groups01:20

Comparing the Survival Analysis of Two or More Groups

Survival analysis is a cornerstone of medical research, used to evaluate the time until an event of interest occurs, such as death, disease recurrence, or recovery. Unlike standard statistical methods, survival analysis is particularly adept at handling censored data—instances where the event has not occurred for some participants by the end of the study or remains unobserved. To address these unique challenges, specialized techniques like the Kaplan-Meier estimator, log-rank test, and Cox...
Methods of Medium Optimization01:28

Methods of Medium Optimization

Optimizing growth media enhances microbial proliferation and maximizes product yield. Statistical experimental design methodologies provide structured and reproducible approaches, offering progressively higher levels of robustness and efficiency.The One-Factor-at-a-Time (OFAT) MethodThe One-Factor-at-a-Time (OFAT) method involves adjusting a single variable while keeping all others constant. However, it cannot detect interactions between variables, often leading to suboptimal outcomes when...
One-Compartment Open Model: Wagner-Nelson and Loo Riegelman Method for ka Estimation01:24

One-Compartment Open Model: Wagner-Nelson and Loo Riegelman Method for ka Estimation

This lesson introduces two critical methods in pharmacokinetics, the Wagner-Nelson and Loo-Riegelman methods, used for estimating the absorption rate constant (ka) for drugs administered via non-intravenous routes. The Wagner-Nelson method relates ka to the plasma concentration derived from the slope of a semilog percent unabsorbed time plot. However, it is limited to drugs with one-compartment kinetics and can be impacted by factors like gastrointestinal motility or enzymatic degradation.
On...
Trial and Error and Algorithm01:12

Trial and Error and Algorithm

A problem-solving strategy is a plan of action used to find a solution. Different strategies have distinct action plans. Trial and error involves trying different solutions until one works. For instance, to fix a broken printer, you might check ink levels, ensure the paper tray isn't jammed, and verify the printer's connection to your laptop. This method can be time-consuming but is commonly used. Thomas Edison, for example, used trial and error to find a suitable filament for the light bulb,...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Fair Innings: An Empirical Test.

Health economics·2025
Same author

Monetary values of increasing life expectancy: Sensitivity to shifts of the survival curve.

Journal of risk and uncertainty·2023
Same author

Consistent valuation of a reduction in mortality risk using values per life, life year, and quality-adjusted life year.

Health economics·2023
Same author

Reappraising the public health benefits conferred by environmental policy: Considerations for an aging society.

Journal of the American Geriatrics Society·2022
Same author

Valuing COVID-19 Morbidity Risk Reductions.

Journal of benefit-cost analysis·2022
Same author

Prevention, treatment, and palliative care: The relative value of health improvements under alternative evaluation frameworks.

Journal of health economics·2022
Same journal

Competition and Collaboration in the AI Race: Country-LevelDirectional Evidence for Risk Monitoring and Policy.

Risk analysis : an official publication of the Society for Risk Analysis·2026
Same journal

Cyber Resilience: Management With Cybersecurity Controls.

Risk analysis : an official publication of the Society for Risk Analysis·2026
Same journal

Jack Fowle: Combining Values, Experience, and Teamwork to Improve Risk Analysis.

Risk analysis : an official publication of the Society for Risk Analysis·2026
Same journal

A Hybrid FMEA-AHP Framework for Risk Prioritization in Nontransparent Artificial Intelligence Systems.

Risk analysis : an official publication of the Society for Risk Analysis·2026
Same journal

Trust-Building Communication for Extreme Heat Preparedness.

Risk analysis : an official publication of the Society for Risk Analysis·2026
Same journal

Spring Broken: A Risk Analysis of Fatal and Nonfatal Traffic Injuries in Florida.

Risk analysis : an official publication of the Society for Risk Analysis·2026
See all related articles

Related Experiment Video

Updated: May 22, 2026

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
08:12

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

Published on: March 1, 2022

Combining experts' judgments: comparison of algorithmic methods using synthetic data.

James K Hammitt1, Yifan Zhang

  • 1Center for Risk Analysis, Harvard University, 718 Huntington Ave., Boston, MA 02115, USA. jkh@harvard.edu

Risk Analysis : an Official Publication of the Society for Risk Analysis
|May 16, 2012
PubMed
Summary
This summary is machine-generated.

Combining expert judgments requires careful method selection. Advanced approaches like copula and frequentist methods outperform simple averaging, especially with varying expert quality and dependency.

More Related Videos

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
07:35

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Published on: October 11, 2018

Related Experiment Videos

Last Updated: May 22, 2026

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
08:12

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

Published on: March 1, 2022

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
07:35

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Published on: October 11, 2018

Area of Science:

  • Decision Analysis
  • Statistics
  • Expert Systems

Background:

  • Expert judgment is a formal method to gather expert opinions on uncertain quantities.
  • Combining multiple experts' subjective probability distributions presents a significant challenge.
  • The equal-weight combination, averaging distributions, is a common but not always optimal approach.

Purpose of the Study:

  • To evaluate and compare the performance of five different methods for combining expert judgments.
  • To assess how expert quality and dependency influence the effectiveness of combination methods.

Main Methods:

  • Simulated expert judgment data were generated to control the underlying distribution process.
  • Five combination methods were evaluated: equal-weight, best-expert, performance, frequentist, and copula.
  • Analyses considered scenarios with two experts of equal/unequal quality and independent/dependent judgments.

Main Results:

  • The copula, frequentist, and best-expert combination methods demonstrated superior performance compared to other approaches.
  • The commonly used equal-weight combination method performed worse than alternatives in the tested scenarios.
  • Performance varied based on expert calibration, quality, and the degree of dependency between judgments.

Conclusions:

  • For combining expert judgments, methods beyond simple averaging, such as copula and frequentist approaches, are recommended.
  • The choice of combination method should consider expert characteristics and the nature of their judgments.
  • Relying solely on equal-weight averaging may lead to suboptimal outcomes in expert elicitation.