Using Survey Questions to Measure Preferences: Lessons from an Experimental Validation in Kenya

Area of Science:

• Behavioral Economics
• Experimental Economics
• Development Economics

Background:

• Economic preferences like time preference, risk attitude, and altruism are fundamental to economic behavior.
• Short survey instruments are increasingly used to measure these preferences globally.
• Previous validation studies focused on university students and representative samples, but experimental validation in low-income populations is less common.

Purpose of the Study:

• To experimentally validate quantitative and qualitative survey measures of economic preferences among low-income individuals in Nairobi, Kenya.
• To assess the cross-cultural and contextual validity of established survey instruments for economic preferences.
• To determine if hypothetical survey tasks and self-assessments accurately predict choices in incentivized experiments within this specific population.

Main Methods:

• An experimental validation study was conducted with low-income individuals in Nairobi, Kenya.
• Participants completed both quantitative (hypothetical experimental tasks) and qualitative (self-assessment) survey measures.
• Choices were compared against actual behavior in incentivized experimental tasks measuring time preference, risk attitude, and altruism.

Main Results:

• Quantitative survey measures of time preference, risk attitude, and altruism significantly predicted choices in incentivized experiments.
• Qualitative, self-assessment survey questions showed no correlation with experimental measures of preferences in the Kenyan sample.
• The findings suggest broad experimental validity for quantitative measures but not for qualitative self-assessments in this context.

Conclusions:

• Quantitative survey instruments, even hypothetical ones, can reliably measure fundamental economic preferences in diverse, low-income settings.
• Qualitative self-assessment questions are not reliable proxies for economic preferences in this population and context.
• Caution is advised when using self-assessment measures for economic preferences in new cultural or socioeconomic contexts without prior validation.