How to Efficiently Annotate Images for Best-Performing Deep Learning-Based Segmentation Models: An Empirical Study...

Area of Science:

• Computer Vision
• Machine Learning
• Data Annotation

Background:

• Deep neural networks (DNNs) excel in image segmentation but require extensive pixel-level annotations.
• Dataset preparation for DNNs is labor-intensive and expensive due to the need for precise object outlines.
• Alternative annotation strategies like weak labels (bounding boxes, scribbles) or noisy labels can reduce costs.

Purpose of the Study:

• To evaluate the cost-effectiveness of various annotation strategies for DNN-based image segmentation.
• To determine if less precise annotations can achieve comparable performance to pixel-level annotations within budget constraints.
• To guide researchers in optimizing annotation budgets for efficient model training.

Main Methods:

• Conducted a cost-effectiveness evaluation of six annotation strategy variants.
• Tested strategies across four diverse datasets.
• Compared performance and cost-efficiency of precise, noisy, and weak annotations.

Main Results:

• The common practice of precise pixel-level annotation is often not the most cost-effective approach.
• Noisy and weak annotations demonstrated usage cases yielding performance similar to perfectly annotated data.
• These alternative methods offered significantly improved cost-effectiveness.

Conclusions:

• Researchers can achieve high-performance DNN image segmentation with significantly reduced annotation costs.
• Utilizing noisy or weak labels is a viable and efficient alternative to precise annotations, especially under budget limitations.
• Findings encourage more efficient use of annotation resources in machine learning research.