Rapid assessment and sample surveys: trade-offs in precision and cost

Health policies in many developing countries are increasingly designed and evaluated by data collected using household surveys. This trend is closely associated with three factors: the increasing complexity of health systems, the increase in pressure for better accountability and improved health system management, and the continuing existence of poorly maintained vital statistics systems. But national household- or individual-level surveys are expensive. These factors have, unsurprisingly, coincided with an increase in the use of rapid survey methods. Although rapid methods have been used frequently, few studies have tested the quality of data obtained by them, and none have compared the cost of the rapid surveys with non-rapid surveys in a developing country context. This paper presents the trade-offs in cost and quality of information obtained from a rapid assessment survey in Ecuador. The results from the rapid survey were compared with results obtained from a national survey conducted six months earlier. The objective was to see what alternative policies might be arrived at if the data from the rapid survey were used in place of the large survey. In addition, the relative costs of obtaining that information were measured. Results indicate that the majority of demographic and contraceptive use measures are similar between the two surveys. The rapid survey was three times as cost-efficient as the traditional survey, if relative bias is not taken into account. However, there appears to be selection bias in the choice of households, which distorts the probability of predicting contraceptive use by residence, and makes it appear that rural women are as likely to use contraception as their counterparts in urban Ecuador. This result has implications for the credibility of rapid surveys, and suggests that their use needs to be tailored for particular programmes and policies, and overseen with, at least, a modicum of statistical caution.