Related Experiment Video
Updated: Jun 28, 2026
10:39
Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
Published on: August 29, 2025
Does speed in completing an online questionnaire have an influence on its reliability?
Christian Montag1, Martin Reuter
1Department of Psychology, University of Bonn, Bonn, Germany. christian.montag@uni-bonn-diff.de
Summary
Offering lottery rewards for online questionnaires does not impact reliability. Study finds participant completion speed, even for fast responders, does not affect Cronbach
Area of Science:
- Psychology
- Psychometrics
- Online Research Methods
Background:
- Online questionnaires often use incentives like lottery rewards to boost participation.
- Concerns exist that rapid completion due to incentives may compromise data reliability.
- Cronbach's alpha is a key measure of internal consistency and questionnaire reliability.
Purpose of the Study:
- To investigate the impact of participant completion speed on questionnaire reliability.
- To determine if offering monetary incentives affects the internal consistency of survey data.
Main Methods:
- Developed a system to measure the time taken by participants to complete online questionnaires.
- Analyzed completion times to identify groups based on response speed.
- Assessed the reliability (Cronbach's alpha) of the Affective Neuroscience Personality Scales (ANPS) across different speed groups.
Main Results:
- Participant speed generally had no significant influence on Cronbach's alpha for personality scales.
- A short lie scale showed a minor exception, but overall reliability remained stable.
- Even extremely fast participants (less than 2 SD below mean time) did not compromise reliability.
Conclusions:
- Incentivized completion speed does not appear to negatively impact the reliability of the ANPS.
- The study suggests that lottery incentives can be used without sacrificing data quality.
- Measures to detect 'cheaters' based on time spent may not be necessary for this scale.
Related Concept Videos
01:29Reliability and Validity
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
02:16Surveys
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
01:28Survey Safety
Surveying near highways, rough terrain, or power lines involves significant risks. Working along highways is particularly dangerous and requires the use of warning signs and flagmen. It is safest to avoid working directly on roads and use offsets whenever possible. When highway work is unavoidable, it must follow all safety guidelines. Surveyors should wear bright clothing, such as orange reflective vests, to ensure visibility to motorists, coworkers, and hunters. In construction zones, wearing...
01:50Cross-Sectional Research
In cross-sectional research, a researcher compares multiple segments of the population at the same time. If they were interested in people's dietary habits, the researcher might directly compare different groups of people by age. Instead of following a group of people for 20 years to see how their dietary habits changed from decade to decade, the researcher would study a group of 20-year-old individuals and compare them to a group of 30-year-old individuals and a group of 40-year-old...
02:20Longitudinal Research
Sometimes we want to see how people change over time, as in studies of human development and lifespan. When we test the same group of individuals repeatedly over an extended period of time, we are conducting longitudinal research. Longitudinal research is a research design in which data-gathering is administered repeatedly over an extended period of time. For example, we may survey a group of individuals about their dietary habits at age 20, retest them a decade later at age 30, and then again...
01:11Statistical Analysis: Overview
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...