Using machine learning and qualitative interviews to design a five-question survey module for women's agency

Area of Science:

• Social Sciences
• Quantitative Research Methods
• Machine Learning Applications

Background:

• Measuring complex concepts in large-scale surveys is challenging with limited questions.
• Qualitative interviews provide rich data but are not feasible for broad surveys.
• Developing concise, reliable survey instruments is crucial for social science research.

Purpose of the Study:

• To propose and validate a new method for designing short survey measures using mixed-methods data and machine learning.
• To create a robust, brief survey module for measuring women's agency.
• To demonstrate the efficacy of machine learning in selecting predictive survey questions.

Main Methods:

• Combined qualitative (semi-structured interviews) and quantitative (close-ended questions) data collection.
• Used qualitative coding to establish a benchmark measure of women's agency.
• Applied machine learning algorithms (LASSO, random forest variants) to identify the most predictive survey questions.

Main Results:

• Developed a five-question index for women's agency that strongly correlates with the benchmark qualitative measure.
• The short index performed comparably to an index using all candidate questions.
• Demonstrated the predictive power of selected close-ended questions for a complex latent construct.

Conclusions:

• The proposed mixed-methods and machine learning approach effectively designs short survey measures for complex concepts.
• This method offers a scalable solution for creating reliable survey instruments in social science.
• The validated women's agency index provides a valuable tool for researchers.