Perspectives on Large Language Models for Relevance Judgment-Reference-Cited by-同舟云学术

Perspectives on Large Language Models for Relevance Judgment

Published:2023-08-09 Issue: Volume: Page:
ISSN:
Container-title:Proceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval
language:
Short-container-title:

Author:

Faggioli Guglielmo¹^ORCID,Dietz Laura²^ORCID,Clarke Charles L. A.³^ORCID,Demartini Gianluca⁴^ORCID,Hagen Matthias⁵^ORCID,Hauff Claudia⁶^ORCID,Kando Noriko⁷^ORCID,Kanoulas Evangelos⁸^ORCID,Potthast Martin⁹^ORCID,Stein Benno¹⁰^ORCID,Wachsmuth Henning¹¹^ORCID

Affiliation:

1. University of Padova, Padova, Italy

2. University of New Hampshire, Durham, NH, USA

3. University of Waterloo, Waterloo, Canada

4. University of Queensland, Brisbane, Australia

5. Friedrich-Schiller-Universität Jena, Jena, Germany

6. Spotify, Delft, Netherlands

7. National Institute of Informatics, Tokio, Japan

8. University of Amsterdam, Amsterdam, Netherlands

9. Leipzig University, Leipzig, Germany

10. Bauhaus-Universität Weimar, Weimar, Germany

11. Leibniz University Hannover, Hannover, Germany

Funder

National Science Foundation

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3578337.3605136

Reference94 articles.

1. Mustafa Abualsaud , Nimesh Ghelani , Haotian Zhang , Mark D. Smucker , Gordon V. Cormack , and Maura R. Grossman . 2018. A System for Efficient High-Recall Retrieval . In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018 , Ann Arbor, MI, USA, July 08--12 , 2018 . ACM, 1317--1320. https://doi.org/10.1145/3209978.3210176 10.1145/3209978.3210176 Mustafa Abualsaud, Nimesh Ghelani, Haotian Zhang, Mark D. Smucker, Gordon V. Cormack, and Maura R. Grossman. 2018. A System for Efficient High-Recall Retrieval. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08--12, 2018. ACM, 1317--1320. https://doi.org/10.1145/3209978.3210176

2. Omar Alonso and Ricardo Baeza-Yates . 2011 . Design and Implementation of Relevance Assessments Using Crowdsourcing. In Advances in Information Retrieval - 33rd European Conference on IR Research , ECIR 2011, Dublin, Ireland, April 18--21, 2011. Proceedings (Lecture Notes in Computer Science , Vol. 6611). Springer, 153-- 164 . https://doi.org/10.1007/978--3--642--20161--5_16 10.1007/978--3--642--20161--5_16 Omar Alonso and Ricardo Baeza-Yates. 2011. Design and Implementation of Relevance Assessments Using Crowdsourcing. In Advances in Information Retrieval - 33rd European Conference on IR Research, ECIR 2011, Dublin, Ireland, April 18--21, 2011. Proceedings (Lecture Notes in Computer Science, Vol. 6611). Springer, 153--164. https://doi.org/10.1007/978--3--642--20161--5_16

3. Omar Alonso and Stefano Mizzaro . 2009 . Can we get rid of TREC assessors ? Using Mechanical Turk for Relevance Assessment. In Proceedings of the SIGIR 2009 Workshop on the Future of IR Evaluation , Vol. 15 . 16. Omar Alonso and Stefano Mizzaro. 2009. Can we get rid of TREC assessors? Using Mechanical Turk for Relevance Assessment. In Proceedings of the SIGIR 2009 Workshop on the Future of IR Evaluation, Vol. 15. 16.

4. Negar Arabzadeh , Maryam Khodabakhsh , and Ebrahim Bagheri . 2021 . BERT-QPP: Contextualized Pre-trained Transformers for Query Performance Prediction. In CIKM '21: The 30th ACM International Conference on Information and Knowledge Management, Virtual Event , Queensland, Australia, November 1 - 5 , 2021. ACM, 2857--2861. https://doi.org/10.1145/3459637.3482063 10.1145/3459637.3482063 Negar Arabzadeh, Maryam Khodabakhsh, and Ebrahim Bagheri. 2021. BERT-QPP: Contextualized Pre-trained Transformers for Query Performance Prediction. In CIKM '21: The 30th ACM International Conference on Information and Knowledge Management, Virtual Event, Queensland, Australia, November 1 - 5, 2021. ACM, 2857--2861. https://doi.org/10.1145/3459637.3482063

5. Unsupervised Question Clarity Prediction through Retrieved Item Coherency

Cited by 32 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Advancing the Search Frontier with AI Agents;Communications of the ACM;2024-08-26

2. Reliable Confidence Intervals for Information Retrieval Evaluation Using Generative A.I.;Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining;2024-08-24

3. Pencils Down! Automatic Rubric-based Evaluation of Retrieve/Generate Systems;Proceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval;2024-08-02

4. Synthetic Test Collections for Retrieval Evaluation;Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval;2024-07-10

5. A Workbench for Autograding Retrieve/Generate Systems;Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval;2024-07-10