Measuring memory is harder than you think: How to avoid problematic measurement practices in memory research

Area of Science:

• Cognitive Psychology
• Neuroscience
• Memory Research

Background:

• Many memory studies use
• old/new
• tasks, which rely on problematic measurement practices.
• Quantifying performance in these tasks is dependent on unknowable distributions of memory signals.
• This has led to diverse and contradictory performance metrics across different memory research fields.

Purpose of the Study:

• To highlight problematic measurement practices in memory research.
• To suggest concrete improvements for memory measurement.
• To advocate for more cumulative and theory-driven memory science.

Main Methods:

• Analysis of common memory measurement tasks, specifically the
• old/new
• task.
• Examination of various performance metrics used in memory research (e.g., A', d', percent correct).
• Proposal of alternative measurement strategies: two-alternative forced-choice tasks and Receiver Operating Characteristic (ROC) analysis.

Main Results:

• The
• old/new
• task's reliance on counterfactual reasoning leads to unreliable performance measures.
• Existing metrics make contradictory and often incorrect assumptions about memory signals.
• Different memory research areas (e.g., visual working memory, eyewitness identification) employ incompatible metrics.

Conclusions:

• Improving measurement practices is crucial for advancing memory science.
• The default memory task should shift to two-alternative forced-choice for performance assessment.
• ROC analysis is recommended over binary
• old/new
• tasks when investigating memory signals or in specific applications like eyewitness identification.