A new look at information retrieval evaluation: proposal for solutions

This is the fourth and the final in the series of the papers on an application of the systems analytic approach to evaluation of information retrieval (IR). In the previous papers terminological and evaluation problems associated with IR were identified and discussed, and it was proposed that the systems analytic approach could provide solutions to these problems. A review of existing systems analytic approaches in IR was given, and a conceptual framework and two models of IR were presented. This paper discusses how the proposed conceptual framework and the models can be used to improve terminology of IR, to analyze the relevance-based method of recall and precision, and to propose new approaches to evaluate relevance relationships and search engines. The proposed evaluation approaches are compared amongst each other and with the existing methods. The paper is concluded by a summary of the contributions of the research and an identification of the directions for a future research in IR evaluation.