Quantifying synergism/antagonism using nonlinear mixed-effects modeling: a simulation study

Cancer drugs are commonly administered in the clinic in the form of mixtures, also called cocktails or combinations. Assessment of synergistic and antagonistic interactions between drugs is therefore an important aspect of cancer research. Numerous methods have been proposed to assess drug interactions, each one based on a null (or additive) model. In this paper, the Loewe additivity index is used as a basis. Parameters of concentration-response curves, used as input to the Loewe equation, are estimated with a nonlinear mixed-effects model. The use of a nonlinear mixed-effects model, in combination with the Loewe index and a procedure to calculate confidence intervals of the index, is referred to as the 'MixLow' method. Simulations of a sham mixture of a drug with itself show that the MixLow method provides parameter estimators that are more precise than those produced by the Median-Effect method, a popular approach that uses log linearization of the concentration-response curves. Coverage of confidence intervals for the interaction index is acceptable for the MixLow method but poor for the Median-Effect method. This is the first comparison report of coverage of confidence intervals for the Loewe interaction index. A mixture of vincristine and topotecan is also analyzed by the MixLow method and the results are compared with those previously published. The MixLow method can be used to quantify drug interactions in any fixed-ratio drug combination study that includes within-group and between-group replicates, and where responses follow a sigmoidal pattern.