Modeling complex measurement error in microbiome experiments to estimate relative abundances and detection effects

Accurate estimates of microbial species abundances are needed to advance our understanding of the role that microbiomes play in human and environmental health. However, laboratory-constructed microbiomes demonstrate that intuitive estimators of microbial relative abundances are biased. To address this, we propose a method to estimate relative abundances, species detection effects, and/or cross-sample contamination in microbiome experiments. We show that certain experimental designs result in identifiable model parameters, and present consistent estimators and asymptotically valid inference procedures that are robust to misspecification of the working likelihood. Notably, our procedure can estimate relative abundances on the boundary of the simplex. We demonstrate the utility of the method for comparing experimental protocols, removing cross-sample contamination, and estimating species' detectability.