A new look at information retrieval evaluation: proposal for solutions.

Area of Science:

• Information Science
• Computer Science
• Systems Analysis

Background:

• Information retrieval (IR) faces challenges in terminology and evaluation.
• Previous work identified these problems and proposed a systems analytic approach.
• Existing IR evaluation methods, like recall and precision, have limitations.

Purpose of the Study:

• To present a conceptual framework and models for IR evaluation using systems analysis.
• To demonstrate how this framework can enhance IR terminology and relevance assessment.
• To propose and compare novel approaches for evaluating relevance and search engines.

Main Methods:

• Application of a systems analytic approach to information retrieval.
• Development of a conceptual framework and two IR models.
• Analysis of relevance-based metrics (recall and precision).
• Proposal and comparative evaluation of new relevance and search engine assessment methods.

Main Results:

• The proposed framework and models offer solutions to IR terminology and evaluation problems.
• New approaches for evaluating relevance relationships and search engines were developed.
• The novel evaluation methods were compared against each other and existing techniques.
• The study provides a comprehensive analysis of the systems analytic approach in IR.

Conclusions:

• The systems analytic approach provides a robust framework for improving IR evaluation.
• The proposed methods offer advancements in assessing relevance and search engine performance.
• This research contributes to the field of information retrieval by offering new evaluation strategies.
• Future research directions in IR evaluation are identified.