Disability Data Collection in a Complex Humanitarian Organisation: Lessons from a Realist Evaluation

Area of Science:

• Disability Studies
• Humanitarian Action
• Development Studies
• Public Policy Evaluation

Background:

• Global focus on disability inclusion has intensified, particularly within the Sustainable Development Goals.
• UN agencies are increasingly prioritizing the measurement of disability inclusion in humanitarian and development efforts.
• The United Nations World Food Programme (WFP) sought to measure assistance to persons with disabilities through its A9 indicator.

Purpose of the Study:

• To evaluate the effects and effectiveness of a specific measurement intervention (A9) for disability inclusion.
• To explore how measurement impacts disability inclusion within a complex humanitarian system.
• To challenge the over-reliance on quantitative data in disability inclusion initiatives.

Main Methods:

• A complexity-informed, realist evaluation methodology was applied.
• A case study of the WFP's A9 measurement intervention was conducted.
• Realist logic of analysis and complexity theory were used to generate context-mechanism-outcome configurations (CMOCs).
• Primary interviews and secondary data were analyzed against the CMOCs.

Main Results:

• The A9 measurement intervention produced delayed, counter-intuitive, and unanticipated effects within the WFP system.
• Path dependency and emergent behaviors influenced the intervention's outcomes.
• Yesterday's intervention mechanisms became tomorrow's implementation context, demonstrating system complexity.

Conclusions:

• The study challenges the current reliance on quantitative data for disability inclusion in humanitarian and development contexts.
• Findings offer insights into leveraging data effectively to enhance disability inclusion.
• Understanding system complexity is crucial for successful measurement interventions in humanitarian aid.