Optimal combination of estimating equations in the analysis of multilevel nested correlated data

Multilevel nested, correlated data often arise in biomedical research. Examples include teeth nested within quadrants in a mouth or students nested within classrooms in schools. In some settings, cluster sizes may be large relative to the number of independent clusters and the degree of correlation may vary across clusters. When cluster sizes are large, fitting marginal regression models using Generalized Estimating Equations with flexible correlation structures that reflect the nested structure may fail to converge and result in unstable covariance estimates. Also, the use of patterned, nested working correlation structures may not be efficient when correlation varies across clusters. This paper describes a flexible marginal regression modeling approach based on an optimal combination of estimating equations. Particular within-cluster and between-cluster data contrasts are used without specification of the working covariance structure and without estimation of covariance parameters. The method involves estimation of the covariance matrix only for the vector of component estimating equations (which is typically of small dimension) rather than the covariance matrix of the observations within a cluster (which may be of large dimension). In settings where the number of clusters is large relative to the cluster size, the method is stable and is highly efficient, while maintaining appropriate coverage levels. Performance of the method is investigated with simulation studies and an application to a periodontal study.