Metacognitive efficiency in learned value-based choice

Area of Science:

• Cognitive Science
• Neuroscience
• Artificial Intelligence

Background:

• Metacognition, the ability to evaluate decision quality, is crucial for higher-order cognition.
• Assessing metacognitive efficiency typically requires normalizing for task performance, which is challenging in dynamic reinforcement learning environments.

Purpose of the Study:

• To develop a novel method for quantifying metacognitive efficiency in reinforcement learning (RL).
• To adapt existing metacognitive efficiency measures for dynamic task difficulties inherent in RL.

Main Methods:

• Repurposed the M-ratio concept using confidence judgments to create a 'Backward model' for assessing metacognitive sensitivity.
• Quantified metacognitive efficiency by comparing the quality of virtual decisions from the Backward model with actual task performance.
• Validated the approach using simulated and empirical data.

Main Results:

• The proposed measure of metacognitive sensitivity, 'Backward performance,' shows comparable properties to existing metrics like quadratic scoring.
• The novel efficiency measure, 'MetaRL.Ratio,' is independent of empirical performance and robust across varying task difficulties.
• The MetaRL.Ratio demonstrates effectiveness in assessing metacognitive efficiency in value-based learning and decision-making.

Conclusions:

• The MetaRL.Ratio offers a promising and reliable tool for evaluating metacognitive efficiency in reinforcement learning.
• This metric overcomes limitations of previous methods in dynamic and complex decision-making scenarios.
• The findings advance our understanding of metacognition in artificial and biological learning systems.