Designing a questionnaire with retrospective pre-post items: Format matters

Area of Science:

• Program evaluation
• Educational research methodology

Background:

• Retrospective Pretest (RPT) is suitable for evaluating one-time educational interventions.
• Existing literature discusses RPT pros and cons but lacks detail on item layout's impact.

Purpose of the Study:

• To investigate how different questionnaire layouts affect Retrospective Pretest (RPT) evaluation outcomes.
• To identify the most effective RPT layout for measuring change in educational interventions.

Main Methods:

• Compared six distinct RPT item layouts with 1941 caregivers across 96 training workshops.
• Assessed layouts based on inattentiveness, unexpected knowledge decline, and degree of before-now change.

Main Results:

• Survey design significantly influences RPT data quality.
• The best-performing layout featured central items with "before" options on the left and "now" options on the right.
• High inattentiveness rates were observed, highlighting a need for better identification and mitigation strategies.

Conclusions:

• Attention to RPT questionnaire layout is crucial for accurate educational intervention evaluation.
• The field needs standardized criteria for assessing survey layout quality.
• Evaluators must proactively address inattentiveness in survey data collection.