Comparative Study of Confidence Intervals for Proportions in Complex Sample Surveys.
Carolina Franco1, Roderick J A Little2, Thomas A Louis3
1Center for Statistical Research and Methodology (CSRM), US Census Bureau, 4600 Silver Hill Road, Washington DC 20233, USA.
Journal of Survey Statistics and Methodology
|August 21, 2019
Summary
For complex surveys, the standard Wald confidence interval (CI) often fails for small proportions. Alternative CIs, like Wilson or Bayes, offer better coverage by improving effective sample size estimation.
Area of Science:
- Statistics
- Survey Methodology
- Computational Statistics
Background:
- The Wald confidence interval (CI) is widely used in complex surveys, including the American Community Survey (ACS).
- This method, adding/subtracting the margin of error (MOE) from a point estimate, frequently results in under-coverage and invalid interval endpoints (e.g., < 0) for small proportions and moderate sample sizes.
- The 'effective sample size,' calculated by dividing the sample size by the design effect, is crucial for CI computation in complex surveys.
Purpose of the Study:
- To evaluate the performance of seven alternative confidence intervals (CIs) compared to the Wald interval for binomial proportions in complex sample surveys.
- To assess the impact of survey design features like clustering, stratification, and varying stratum sampling fractions on CI coverage and width.
- To investigate methods for improving the estimation of effective sample size to enhance CI calibration.
Main Methods:
- Simulations were conducted to assess the coverage and width of various CIs.
- The study considered complex survey designs, including clustering and stratification.
- Effective sample size estimation was improved using superpopulation modeling.
Main Results:
- The standard Wald interval demonstrated marked under-coverage, especially with clustering and when using simple design-based variance estimators.
- All tested intervals showed under-coverage issues under certain complex survey conditions.
- Improving effective sample size estimation via superpopulation modeling significantly enhanced the calibration of alternative CIs.
Conclusions:
- The Wald interval is inadequate for small proportions in complex surveys.
- Alternative intervals, specifically the Wilson and Bayes uniform prior intervals, are recommended for improved performance.
- The Jeffreys prior interval also showed promising results, performing nearly as well as the recommended alternatives.
Related Concept Videos
01:21Confidence Intervals
10.2K
An unbiased point estimate is often insufficient to predict a population estimate, such as population mean or population proportion. In this scenario, a confidence interval is used. A confidence interval is an estimate similar to a sample proportion. However, unlike the point estimate which is a single value, the confidence interval contains a range of values. These values have lower and upper limits, known as confidence limits, and can be designated as L1 and L2, respectively.
A...
A...
10.2K
00:54Uncertainty: Confidence Intervals
10.2K
The confidence interval is the range of values around the mean that contains the true mean. It is expressed as a probability percentage. The interpretation of a 95% confidence interval, for instance, is that the statistician is 95% confident that the true mean falls within the interval. The upper and lower limits of this range are known as confidence limits. The confidence limits for the true mean are estimated from the sample's mean, the standard deviation, and the statistical factor...
10.2K
01:19Interpretation of Confidence Intervals
9.4K
A confidence interval is a better estimate of the population than a point estimate, as it uses a range of values from a sample instead of a single value.
Confidence intervals have confidence coefficients that are crucial for their interpretation. The most common confidence coefficients are 0.90, 0.95, and 0.99, which can be written as percentages–90%, 95%, and 99%, respectively.
Suppose a person calculates a confidence interval with a confidence coefficient of 0.95. In that case, they can...
Confidence intervals have confidence coefficients that are crucial for their interpretation. The most common confidence coefficients are 0.90, 0.95, and 0.99, which can be written as percentages–90%, 95%, and 99%, respectively.
Suppose a person calculates a confidence interval with a confidence coefficient of 0.95. In that case, they can...
9.4K
01:20Sample Proportion and Population Proportion
6.4K
Collecting samples or responses from an entire population takes significant time and effort, so a researcher collects responses from only a sample of that population. Suppose a study needs to collect information about a specific mobile application. After sample collection, the researcher analyzes the data and discovers that most individuals in the sample use that specific mobile application. The sample proportion measures the number of individuals in a sample who either use or don't use the...
6.4K
01:25Confidence Interval for Estimating Population Mean
8.8K
A point estimate of the population mean is obtained from a single sample. Such a point estimate does not represent a population well because it needs to account for variability in the population. Single point estimate can also be biased despite the sample being selected randomly. Thus, a point estimate is often unreliable. A confidence interval is needed to reduce this unreliability.
A confidence interval for the mean is a range of values that provides an estimate of the population mean. As the...
A confidence interval for the mean is a range of values that provides an estimate of the population mean. As the...
8.8K
02:16Surveys
16.6K
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
16.6K