Affiliation:
1. University of Glasgow, Glasgow, United Kingdom
Abstract
The setting of the term frequency normalization hyper-parameter suffers from the query dependence and collection dependence problems, which remarkably hurt the robustness of the retrieval performance. Our study in this article investigates three term frequency normalization methods, namely normalization 2, BM25's normalization and the Dirichlet Priors normalization. We tackle the query dependence problem by modifying the query term weight using a Divergence From Randomness term weighting model, and tackle the collection dependence problem by measuring the correlation of the normalized term frequency with the document length. Our research hypotheses for the two problems, as well as an automatic hyper-parameter setting methodology, are extensively validated and evaluated on four Text REtrieval Conference (TREC) collections.
Publisher
Association for Computing Machinery (ACM)
Subject
Computer Science Applications,General Business, Management and Accounting,Information Systems
Reference16 articles.
1. Amati G. 2003. Probabilistic models for information retrieval based on divergence from randomness. Ph.D. dissertation Department of Computing Science University of Glasgow. Amati G. 2003. Probabilistic models for information retrieval based on divergence from randomness. Ph.D. dissertation Department of Computing Science University of Glasgow.
2. Probabilistic models of information retrieval based on measuring the divergence from randomness
3. DeGroot M. 1989. Probability and Statistics 2nd edition ed. Addison Wesley Reading MA. DeGroot M. 1989. Probability and Statistics 2nd edition ed. Addison Wesley Reading MA.
Cited by
12 articles.
订阅此论文施引文献
订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献
1. A topic‐based term frequency normalization framework to enhance probabilistic information retrieval;Computational Intelligence;2020-05
2. A New Term Frequency Normalization Model for Probabilistic Information Retrieval;The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval;2018-06-27
3. Bayesian Optimization for Optimizing Retrieval Systems;Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining;2018-02-02
4. Divergence-from-Randomness Models;Encyclopedia of Database Systems;2018
5. Divergence-from-Randomness Models;Encyclopedia of Database Systems;2017