QADI as a New Method and Alternative to Kappa for Accuracy Assessment of Remote Sensing-Based Image Classification

Area of Science:

• Remote Sensing and Geospatial Analysis
• Image Processing and Computer Vision

Background:

• Image classification accuracy is crucial for reliable analysis, often assessed using metrics like the Kappa index.
• The traditional Kappa index faces criticism for its sensitivity to asymmetric class distributions, limiting its reliability.
• Existing alternatives to Kappa have shown limited practical success in improving accuracy assessment.

Purpose of the Study:

• To introduce a novel accuracy assessment index, the Quantity and Allocation Disagreement Index (QADI).
• To address the limitations of the Kappa index, particularly its sensitivity to asymmetric distributions.
• To provide a more reliable measure for evaluating image classification accuracy compared to existing methods.

Main Methods:

• Developed the Quantity and Allocation Disagreement Index (QADI) to quantify disagreement between classified and reference maps.
• QADI computes wrongly labeled pixels (A) and differences in class pixel counts (Q) to determine disagreement.
• Compared the performance of QADI against the Kappa index across six diverse use cases.

Main Results:

• The QADI index demonstrated superior reliability in classification accuracy assessments compared to the Kappa index.
• QADI is not sensitive to asymmetric distributions, offering a more robust evaluation.
• A toolbox for implementing QADI within a GIS software environment was developed.

Conclusions:

• The novel QADI index provides a more dependable method for assessing image classification accuracy.
• QADI's ability to handle asymmetric distributions makes it a valuable improvement over the Kappa index.
• The developed GIS toolbox facilitates the practical application of QADI in geospatial analysis.