Comparison of different maximum likelihood estimators in a small sample logistic regression with two independent...

Area of Science:

• Biostatistics
• Statistical modeling
• Epidemiology

Background:

• Bias in parameter estimation is critical in logistic regression, especially with small sample sizes.
• Previous work by Walter explored bias in log odds ratio estimates for 2x2 tables.
• Understanding parameter bias is essential for accurate interpretation of binary outcome associations.

Purpose of the Study:

• To compute and compare the distributions of estimated logistic regression parameters (b1 and b2) for two independent binary variables in small samples.
• To evaluate six different maximum likelihood estimation methods.
• To identify the optimal estimation method based on specific criteria.

Main Methods:

• Simulation of small sample logistic regressions with two independent binary predictors.
• Computation of parameter distributions using six distinct maximum likelihood estimation techniques.
• Comparison of estimated parameter values against true parameter values.

Main Results:

• The performance of estimation methods varied based on the frequency of the outcome of interest.
• The choice between minimizing bias or mean square error influenced the selection of the best method.
• Distributions of estimated parameters b1 and b2 were characterized for small sample sizes.

Conclusions:

• No single maximum likelihood estimation method is universally superior for small sample logistic regression.
• The optimal method is contingent upon the specific data characteristics (outcome frequency) and the researcher's priority (bias vs. mean square error).
• This analysis provides guidance for selecting appropriate estimation strategies in small sample logistic regression settings.