On the effect of relevance scales in crowdsourcing relevance assessments for Information Retrieval evaluation-Reference-Cited by-同舟云学术

On the effect of relevance scales in crowdsourcing relevance assessments for Information Retrieval evaluation

Published:2021-11 Issue:6 Volume:58 Page:102688
ISSN:0306-4573
Container-title:Information Processing & Management
language:en
Short-container-title:Information Processing & Management

Author:

Roitero Kevin^ORCID,Maddalena Eddy,Mizzaro Stefano^ORCID,Scholer Falk^ORCID

Publisher

Elsevier BV

Subject

Library and Information Sciences,Management Science and Operations Research,Computer Science Applications,Media Technology,Information Systems

Reference63 articles.

1. Analysis of ordinal categorical data;Agresti,2010

2. Games with a purpose;von Ahn;Computer,2006

3. Evaluating topic representations for exploring document collections;Aletras;Journal of the Association for Information Science and Technology,2017

4. Alonso, O., & Mizzaro, S. (2009). Can we get rid of TREC assessors? Using Mechanical Turk for relevance assessment. In Proceedings of the SIGIR 2009 workshop on the future of IR evaluation, Vol. 15 (p. 16).

5. Using crowdsourcing for TREC relevance assessment;Alonso;Information Processing & Management,2012

Cited by 13 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Crowdsourced Fact-checking: Does It Actually Work?;Information Processing & Management;2024-09

2. What Matters in a Measure? A Perspective from Large-Scale Search Evaluation;Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval;2024-07-10

3. Enhancing Human Annotation: Leveraging Large Language Models and Efficient Batch Processing;Proceedings of the 2024 ACM SIGIR Conference on Human Information Interaction and Retrieval;2024-03-10

4. Does high-quality news attract engagement on social media? Mediatization, media logic, and the contrasting values that shape news sharing, liking, and commenting on Facebook;New Media & Society;2024-02-07

5. Reliable Information Retrieval Systems Performance Evaluation: A Review;IEEE Access;2024