THE SHANNON MODEL OF IR SYSTEMS

Abstract

This note was evoked by the reference by Karen Sparck Jones to a paper by Zunde and Slamecka which has recently been reprinted in Introduction to Information Science, edited by Saracevic. Zunde and Slamecka purport to show that, for optimum performance of IR systems, the frequency distribution of descriptor terms should conform with a geometric progression. This result is at variance with the widely accepted result derived from the Shannon model which shows that optimum performance of an IR system occurs when the descriptor terms are equi‐probable, i.e. when their frequency distribution is uniform. The uncertainty arising from these two different solutions to the same problem clearly led Karen Sparck Jones to have some reservations about the theoretical justification for her interesting idea of weighting search terms to give them, in effect, the equal weights that the usual Shannon result demands for optimum performance. But Sparck Jones need have no such reservations. The result obtained by Zunde and Slamecka, though plausible because it has some fortuitous semblance to the distributions of terms found in real systems, is in fact erroneous.