Effect of changing the radiation dose range on the in vitro cytogenetic dose response to gamma-rays

To examine the distortion of the linear quadratic (LQ) model of in vitro cytogenetic dose response over an extended range of γ-ray doses by analyzing the available literature data, and to establish the dose ranges, in which the LQ dose response curve (DRC) can be most accurately fitted for biological dosimetry.

Data on yields of dicentrics (Dic) or dicentrics plus centric rings (Dic + CR) induced in vitro in human lymphocytes by acute γ-rays were extracted from 108 open sources. The overall dose response dataset in the dose range up to 50 Gy was fitted to a fractional-rational (FR) model, which included a 'basic' LQ function in the numerator, and a reduction factor dependent on the square of the dose in the denominator. Cytogenetic dose response data obtained at Grigoriev Institute for Medical Radiology, Kharkiv, Ukraine (GIMRO) in the range 0.1 - 20.3 Gy acute γ-rays were fitted to the LQ model with the progressive changing minimum or maximum radiation dose.

The overall dose response, as expected, followed the LQ function in the dose range ≤5 Gy, but in the extended dose range appeared to be S-shaped, with intensive saturation and a plateau at doses ≥22 Gy. Coefficients of the 'basic' LQ equation in FR model were very close to many published DRCs; calculated asymptote was 17. Fitting of the GIMRO dataset to the LQ model with the shift of the dose range showed the increase in linear coefficient with the increment of either minimum or maximum radiation dose, while the decline of the quadratic coefficient was regulated mostly by the increase of the highest dose. The best goodness of fit, assessed by lower χ² values, occurred for dose ranges 0.1 - 1.0 Gy; 0.5 - 5.9 Gy; 1.0 - 7.8 Gy; 2.0 - 9.6 Gy, 3.9 - 16.4 Gy and 5.9 - 20.3 Gy. The 'see-saw' effect in changes of LQ coefficients was confirmed by re-fitting datasets published by other laboratories.

The classical LQ model with fixed coefficients appears to have limited applicability for cytogenetic dosimetry at radiation doses >5 Gy due to the saturation of the dose response. Different response of the LQ coefficients to the changes of the dose range must be considered during the DRC construction. Proper selection of minimum and maximum dose in calibration experiments makes it possible to improve the goodness of fit of the LQ DRC.