Evaluating Iterative Deep Learning as a Labeling-Efficient Strategy for Tubular Segmentation in Digital...

Chronic kidney disease (CKD) is a prevalent condition worldwide and a significant global health burden that is expected to increase in the coming decades. Morphological evaluation of renal tubules is critical for diagnosis and prognosis; however, manual annotation is labor-intensive and time-consuming. Automatic AI-based segmentation offers a promising solution, yet it still depends on extensive manual annotations and diverse datasets, which are difficult to establish. One potential strategy to mitigate this burden is iterative, model-assisted annotation, where models are progressively improved through human-in-the-loop corrections. The current study addresses the need for rigorous evaluation of annotation efficiency in such workflows. It investigates how annotation speed-up can be quantified in a consistent and informative manner, using renal tubules with high morphological variability as a use case. Complementary strategies for measuring annotation efficiency were applied to Quick Annotator (QA), an interactive deep-learning tool for iterative annotation. QA models were trained with 5-, 10-, and 20-min annotation intervals, followed by refinement on data with diverse tubular lesions. Annotation efficiency was quantified using multiple speed-up metrics and regression-based modeling, while segmentation quality was assessed to ensure practical relevance. QA improved annotation efficiency, particularly in the 10- and 20-min workflows, reducing annotation time by up to a factor of five compared to manual annotation in QuPath. Notably, speed-up estimates varied across evaluation methods, emphasizing the importance of appropriate metric selection and interpretation. Segmentation performance was comparable to reference models in some settings but lower for complex, diseased tubules.