Measuring changes with traditional and retrospective pre-posttest self-report surveys for a brief intervention...

Area of Science:

• Program Evaluation
• Educational Research
• Social Sciences

Background:

• Evaluating brief intervention programs requires accurate measurement of participant change.
• Traditional pre-posttest surveys may be limited by ceiling effects when participants have high initial beliefs.

Purpose of the Study:

• To compare the effectiveness of retrospective and traditional pre-posttest surveys in evaluating a brief intervention.
• To identify which survey method better captures changes in participant understanding, practices, and beliefs.

Main Methods:

• The Latino Enrollment Institute (LEI) program at the University of Notre Dame was evaluated.
• Both traditional and retrospective pre-posttest self-report surveys were administered to participants.

Main Results:

• The retrospective survey approach demonstrated a greater magnitude of perceived change in participants' understanding and practices.
• Retrospective designs detected changes in beliefs that traditional pre-posttest surveys missed due to pretest ceiling effects.

Conclusions:

• Retrospective survey designs are valuable for evaluating interventions, particularly when participants self-select and possess high initial beliefs.
• Evaluators should consider retrospective designs to overcome limitations of traditional methods and capture nuanced changes in beliefs and practices.