How to complete performance graphs in content-based image retrieval: add generality and normalize scope.

Area of Science:

• Computer Science
• Information Retrieval
• Machine Learning

Background:

• Traditional Content-Based Image Retrieval (CBIR) system performance evaluation using Precision-Recall or Precision-Scope graphs provides an incomplete view.
• Existing methods obscure the impact of irrelevant items (embeddings) on retrieval performance.

Purpose of the Study:

• To propose a comprehensive and normalized method for describing CBIR system ranking performance.
• To address the limitations of current evaluation metrics by accounting for irrelevant items and providing a more complete performance overview.
• To introduce novel visualization tools for a thorough analysis of retrieval performance.

Main Methods:

• Developed a normalized description of ranking performance by comparing against an Ideal Retrieval System.
• Advocated for normalization concerning relevant class size and specific normalized scope values (number of retrieved items).
• Proposed new three and two-dimensional performance graphs for comprehensive total recall studies across various embeddings.

Main Results:

• The proposed method offers a more complete overview of CBIR system performance compared to traditional Precision-Recall or Precision-Scope graphs.
• Normalization techniques effectively account for the influence of irrelevant items (embeddings).
• New graphical representations facilitate detailed total recall studies.

Conclusions:

• The proposed comprehensive and normalized evaluation framework enhances the understanding of Content-Based Image Retrieval system performance.
• The new visualization methods provide deeper insights into retrieval effectiveness, particularly concerning irrelevant items and total recall.
• This work contributes to more robust and informative performance assessment in the field of image retrieval.