Inferring the impact of humanitarian responses on population mortality: methodological problems and proposals

Reducing excess population mortality caused by crises due to armed conflict and natural disasters is an existential aim of humanitarian assistance, but the extent to which these deaths are averted in different humanitarian responses is mostly unknown. This information gap arguably weakens governance and accountability. This paper considers methodological challenges involved in making inferences about humanitarian assistance's effect on excess mortality, and outlines proposed approaches. Three possible measurement questions, each of which contributes some inferential evidence, are presented: (1) whether mortality has remained within an acceptable range during the crisis (for which different direct estimation options are presented); (2) whether the humanitarian response is sufficiently appropriate and performant to avert excess mortality (a type of contribution analysis requiring in-depth audits of the design of humanitarian services and of their actual availability, coverage and quality); and (3) the actual extent to which humanitarian assistance has reduced excess deaths (potentially the most complex question to answer, requiring application of causal thinking and careful specification of the exposure, and for which either quasi-experimental statistical modelling approaches or a combination of verbal and social autopsy methods are proposed). The paper concludes by considering possible 'packages' of the above methods that could be implemented at different stages of a humanitarian response, and calls for investment in improved methods and actual measurement.