Inspecting Retrieval Engines Based on Term’s Weight

Abstract

We introduce a revised way of inspecting retrieval engines based on the term’s weight which we call it as discrimination power of a term. The aim of this methodology is to make engine developers find the indexing or query processing components that are problematic in ranking retrieved documents. The internal processes of indexing and query processing are modeled in a Bayesian inference network. The nodes in this model can be terms, phrases, sentences, documents, etc. The layers represent internal step-by-step processing components like stemming, tokenizing, POS Tagging, parsing, etc. By computing and showing the extent to which each node plays a role in causing a relevant (or irrelevant) document to be ranked high (or low), the proposed method can signal the nodes to be examined and hence the components to be modified. We insist that the proposed method can be used for automatic failure analysis of a system with test runs and that its effectiveness can be improved even automatically.