Related Experiment Video
Updated: May 23, 2026
09:43
The 5-Choice Serial Reaction Time Task: A Task of Attention and Impulse Control for Rodents
Published on: August 10, 2014
Identifying careless responses in survey data.
Adam W Meade1, S Bartholomew Craig
1Department of Psychology, North Carolina State University, Campus Box 7650, Raleigh, NC 27695-7650, USA. awmeade@ncsu.edu
Psychological Methods
|April 18, 2012
Summary
Detecting careless responses in online surveys is crucial for data quality. This study identified two patterns of careless responding and recommended specific methods to ensure reliable data collection from participants.
Area of Science:
- Psychological Methods
- Survey Methodology
- Data Quality Assurance
Background:
- Anonymous internet surveys, especially with mandatory participation, raise concerns about data quality.
- Limited guidance exists on detecting careless responses in research data.
- Previous methods for identifying careless respondents lacked comprehensive examination of their relationships and identified data patterns.
Purpose of the Study:
- To examine and compare various methods for detecting careless responses in online surveys.
- To investigate the relationships among different indicators of careless responding.
- To identify distinct patterns of careless response and the indices effective in detecting them.
Main Methods:
- Utilized two studies, including analysis of real survey data and simulation of known random response patterns.
- Evaluated methods such as special detection items, response consistency indices, multivariate outlier analysis, response time, and self-reported diligence.
- Examined the efficacy of different indices based on the nature of the data and response patterns.
Main Results:
- Identified two distinct patterns of careless response: random and nonrandom.
- Different indices are required to effectively detect these distinct response patterns.
- Approximately 10%-12% of undergraduate students in a course credit survey were identified as careless responders.
- The efficacy of detection indices was significantly influenced by the data's characteristics.
Conclusions:
- Recommends using identified (non-anonymous) responses for better data quality control.
- Suggests incorporating instructed response items and employing consistency indices and multivariate outlier analysis.
- Highlights the need for tailored strategies to address different types of careless responding in online surveys.
Related Concept Videos
02:16Surveys
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
01:07Data Collection by Survey
The systematic method of obtaining and analyzing accurate information of a population is called data collection. A survey is a standard method of data collection that involves collecting information from a target human population about their experience, opinion, or knowledge of a product, service, or process. The responses are recorded and interpreted. The most common survey examples are written questionnaires, face-to-face or telephonic conversations, focus groups, and electronic (e-mail or...
01:00Detection of Gross Error: The Q Test
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
01:19Errors and Mistakes in Surveying
Errors and mistakes in surveying refer to inaccuracies in measurements and data recording. The errors are deviations from the actual value caused by human sensory limitations, equipment flaws, or environmental effects. These errors are typically unintentional and can result from the inherent imperfections in the instruments used, atmospheric conditions, or the observer’s inability to perceive exact measurements. On the other hand, mistakes are caused by the surveyor's lack of attention,...
01:15Systematic Error: Methodological and Sampling Errors
In the case of systematic errors, the sources can be identified, and the errors can be subsequently minimized by addressing these sources. According to the source, systematic errors can be divided into sampling, instrumental, methodological, and personal errors.
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
01:09Censoring Survival Data
Survival analysis is a statistical method used to analyze time-to-event data, often employed in fields such as medicine, engineering, and social sciences. One of the key challenges in survival analysis is dealing with incomplete data, a phenomenon known as "censoring." Censoring occurs when the event of interest (such as death, relapse, or system failure) has not occurred for some individuals by the end of the study period or is otherwise unobservable, and it might have many different reasons...