Effect of heterogeneous radiosensitivity on the optimal fractionation in radiotherapy

to determine optimal fractionation schedule which provides maximum biologically effective dose (BED) in the targeted tumor in the case of heterogeneous radiosensitivity of malignant cells in combination with a spatially varying sparing factor for the organ at risk.

the mathematical framework used in the current study is based on the linear-quadratic (LQ) model with heterogeneous parameters alpha and beta for malignant cells. To determine optimal fractionation, we consider changes in BED in the treatment target (BED_tar) under the condition of fixed BED in the affected normal tissue (BED_nt).

we demonstrate both analytically and via numerical calculations that there exists an optimal number of fractions for which biologically effective dose in the uniformly irradiated target is at maximum. It is also shown that dependence of BED_tar on number of fractions is generally non-monotonic and is affected by the variances σ_α and σ_β of the alpha and beta radiosensitivities, respectively. In a particular case when σ_α and σ_β are sufficiently small, expression for the optimal target dose which maximizes BED_tar was derived analytically.

the obtained results demonstrate that in the presence of heterogeneous alpha and beta in the tumor, hypofractionation can either increase or decrease BED_tar depending on the variances σ_α and σ_β. Consequently, intratumor heterogeneity is an important factor which can affect radiobiological comparison of different fractionation regimens.