Proposal for a Five-Step Method to Elicit Expert Judgment

Area of Science:

• Psychological Research
• Statistical Analysis
• Bayesian Frameworks

Background:

• Expert knowledge elicitation is crucial for statistical analysis.
• Directly obtaining distributional representations from experts is often unsatisfactory.
• Feedback mechanisms can improve the quality of expert elicitation.

Purpose of the Study:

• To propose a novel Five-Step Method for expert knowledge elicitation.
• To decompose the elicitation process into smaller, manageable steps with feedback.
• To represent expert tacit knowledge and uncertainty using probabilistic parameters.

Main Methods:

• The Five-Step Method involves iterative elicitation and feedback.
• Location, scale, and shape parameters are elicited sequentially.
• Interactive software facilitates feedback and adjustment of expert input.
• User feasibility and internal validity were assessed through three studies.

Main Results:

• The Five-Step Method allows experts to refine their probabilistic representations.
• Feedback enhances the accuracy and reliability of elicited expert knowledge.
• The method is feasible and internally valid for practical application.

Conclusions:

• The proposed Five-Step Method offers a structured approach to expert elicitation.
• This method improves the incorporation of expert prior knowledge in statistical analyses.
• It provides a robust way to calibrate probability distributions and assess prior-data conflict.