Assessing the agreement of radiomic tools for dosiomics: A multi-software comparative study

Radiomics involves extracting and analyzing quantitative imaging features to support medical decision-making, particularly in radiology and oncology. When applied to radiotherapy dose distributions, this approach, referred to as 'dosiomics', aims to identify the spatial dose patterns associated with treatment outcomes. However, software discrepancies in feature extraction may hinder reproducibility and limit the clinical adoption of radiomic/dosiomic models.

This study presents the first comprehensive evaluation of software agreement and feature reproducibility across tools in the field of dosiomics, assessing seven feature-extraction tools. The evaluation focused on the impact of built-in image pre-processing steps (e.g., interpolation and discretization), feature-extraction configurations (i.e., aggregation methods), and the morphological characteristics of the regions of interest (ROIs), such as the presence of holes or disconnected components.

Five open-source programs (MIRP, S-IBEX, RaCaT, SERA, and PyRadiomics) and two proprietary tools (SPAARC and RadiomiCRO) were evaluated. The Image Biomarker Standardization Initiative (IBSI) digital phantom was used to preliminarily assess software IBSI-compliance and to identify and exclude features with inconsistent implementation from subsequent analyses. Dosiomic features were then extracted from a digital dataset comprising eight Intensity Modulated Radiation Therapy (IMRT) dose distributions emulating a head and neck radiotherapy plan (available in both isotropic and anisotropic formats) and 10 ROIs, following a systematic feature extraction framework. The effects of pre-processing parameters, feature-extraction configurations, and ROI morphological characteristics were analyzed systematically. The evaluation metrics included the percentage of matching features across software to the third significant digit, the Agreement metric, and the coefficient of variation (CV) to quantify both software performance and dosiomic feature variability across them.

The preliminary IBSI-compliance evaluation showed that MIRP, S-IBEX, RaCaT, and SERA achieved over 94% matching features with IBSI benchmark values. In contrast, SPAARC, RadiomiCRO, and PyRadiomics demonstrated lower compliance due to non-computable features. On dose distributions, all tools exhibited high match percentages (>77%) for the isotropic dataset, which did not require software-specific interpolation. However, discrepancies increased significantly with program-specific interpolation for the anisotropic dose dataset, with match rates dropping to 14%. Agreement across software was consistently high for the isotropic dataset but notably lower for the anisotropic dataset. This trend was less evident when looking at the CV, which showed only a mild increase for the anisotropic format. Fixed bin size (FBS) discretization displayed lower Agreement and higher CV values, particularly in the cumulative intensity-volume histogram (IVH) feature family. High CV values were predominantly observed for some feature family-ROI combinations, including GLRLM, GLSZM, and NGLDM computed using 2.5D/3D aggregation methods. Additionally, we observed that some binary masks were incorrectly generated (e.g., without holes) when using the DICOM format, therefore, we relied on NRRD input files whenever possible, resulting in feature reproducibility remaining unaffected by this aspect.

The findings of this study indicate that, when properly configured, the tools show good overall agreement, with variability limited to specific features and pre-processing choices. While variations in program-specific resampling and FBS discretization implementation are present, their overall impact on dosiomic feature reproducibility remains minimal.