A cognitive model for aggregating people's rankings

Area of Science:

• Cognitive Science
• Computational Psychology
• Data Science

Background:

• Combining individual judgments into a collective decision is a fundamental challenge.
• Classic theories of knowledge representation and judgment offer insights into how individuals rank items.
• Existing methods for aggregating rankings may not fully capture underlying cognitive processes.

Purpose of the Study:

• To develop a cognitive modeling approach for combining people's rankings of items.
• To infer an aggregate ranking and measures of individual expertise from behavioral data.
• To evaluate the model's performance against traditional methods and the
• wisdom of the crowd
• effect.

Main Methods:

• Developed a cognitive model based on psychological theories of knowledge representation.
• Implemented the cognitive model as a Bayesian graphical model.
• Utilized computational sampling for inferring aggregate rankings and individual expertise.

Main Results:

• The model successfully produced aggregate rankings close to ground truth across 23 datasets.
• The model's aggregate rankings generally outperformed individual judgments, demonstrating the "wisdom of the crowd" effect.
• The model demonstrated superior performance compared to the traditional Borda count method and accurately inferred relative expertise.

Conclusions:

• Cognitive modeling offers a powerful approach for analyzing and combining ranking data.
• The developed model provides a robust method for aggregating judgments and assessing expertise.
• This approach has significant implications for "wisdom of the crowd" research and applications involving collective intelligence.