Mini-batch Estimation for Deep Cox Models: Statistical Foundations and Practical Guidance

Area of Science:

• Machine Learning
• Statistical Modeling
• Survival Analysis

Background:

• Stochastic Gradient Descent (SGD) is commonly used for optimizing deep Cox neural networks (Cox-NN).
• SGD updates model parameters using mini-batches, optimizing the average mini-batch partial-likelihood, which differs from the standard partial-likelihood.

Purpose of the Study:

• To develop and analyze the statistical properties of the mini-batch maximum partial-likelihood estimator (mb-MPLE) for Cox-NN.
• To provide theoretical guarantees and practical guidance for using SGD in Cox models.

Main Methods:

• Theoretical analysis of the mini-batch maximum partial-likelihood estimator (mb-MPLE).
• Establishing consistency and convergence rates for mb-MPLE in Cox-NN and Cox regression.
• Characterizing SGD dynamics, including the impact of learning rate and batch size ratios.

Main Results:

• mb-MPLE for Cox-NN is consistent and achieves optimal minimax convergence rates.
• For linear Cox regression, mb-MPLE is $\sqrt{n}$-consistent and asymptotically normal, with variance approaching the information lower bound as batch size increases.
• The ratio of learning rate to batch size is critical for SGD in Cox-NN; SGD iterative convergence is characterized for Cox regression.

Conclusions:

• The developed mb-MPLE provides a statistically sound and efficient method for optimizing Cox models using SGD.
• Theoretical insights and practical guidance are offered for hyperparameter tuning and ensuring convergence.
• mb-MPLE demonstrates effectiveness in large-scale applications where standard methods are intractable.