Related Experiment Video
Updated: Jul 15, 2026
05:58
Using Rapid Serial Visual Presentation to Measure Set-Specific Capture, a Consequence of Distraction While Multitasking
Published on: August 29, 2018
Identifying careless responding in ecological momentary assessment: Inconsistent signals from different detection
Leonie V D E Vogelsmeier1, Björn S Siepe2, Esther Ulitzsch3
1Department of Methodology and Statistics, Tilburg University.
Psychological Methods
|July 13, 2026
Summary
Detecting careless responding in intensive longitudinal data is crucial for psychological science. This study compares 13 methods, finding significant variation and highlighting the need for further empirical validation of these important tools.
Area of Science:
- Psychological science
- Methodology
- Data quality
Background:
- Intensive longitudinal data, such as ecological momentary assessment (EMA), are vital for understanding psychological dynamics.
- Careless responding in EMA studies can significantly degrade data quality and invalidate research findings.
- Existing data-driven methods for detecting careless responding are numerous and complex, posing a challenge for researchers.
Purpose of the Study:
- To summarize, apply, and compare 13 distinct methods for flagging careless responses in intensive longitudinal data.
- To provide a practical guide and supplementary tutorials for researchers to evaluate and report careless responding in their own EMA data.
- To assess the current state of methods for detecting careless responding and identify promising approaches.
Main Methods:
- Comparison of 13 data-driven methods for detecting careless responding.
- Application of these methods to a large ecological momentary assessment (EMA) dataset from the WARN-D study (n = 1,194 participants, 206,318 observations).
- Evaluation of the consistency and divergence of results across the different detection methods.
Main Results:
- Significant variability was observed in the results produced by the 13 compared methods for flagging careless responses.
- The effectiveness and agreement between different careless responding detection approaches varied considerably.
- The study underscores that methods for detecting careless responding are still in early developmental stages.
Conclusions:
- The current landscape of careless responding detection methods is diverse but lacks consensus, producing highly divergent results.
- Researchers are provided with recommendations on the most promising methods based on the study's findings.
- Further empirical validation is essential for advancing the reliability and utility of careless responding detection techniques in psychological research.
