Report on the Dagstuhl Seminar on Frontiers of Information Access Experimentation for Research and Education
-
Published:2023-06
Issue:1
Volume:57
Page:1-28
-
ISSN:0163-5840
-
Container-title:ACM SIGIR Forum
-
language:en
-
Short-container-title:SIGIR Forum
Author:
Bauer Christine1, Carterette Ben2, Ferro Nicola3, Fuhr Norbert4, Beel Joeran, Breuer Timo, Clarke Charles L. A., Crescenzi Anita, Demartini Gianluca, Di Nunzio Giorgio Maria, Dietz Laura, Faggioli Guglielmo, Ferwerda Bruce, Fröbe Maik, Hagen Matthias, Hanbury Allan, Hauff Claudia, Jannach Dietmar, Kando Noriko, Kanoulas Evangelos, Knijnenburg Bart P., Kruschwitz Udo, Li Meijie, Maistro Maria, Michiels Lien, Papenmeier Andrea, Potthast Martin, Rosso Paolo, Said Alan, Schaer Philipp, Seifert Christin, Spina Damiano, Stein Benno, Tintarev Nava, Urbano Julián, Wachsmuth Henning, Willemsen Martijn C., Zobel Justin
Affiliation:
1. Paris Lodron University Salzburg, Austria 2. University of Delaware and Spotify, United States 3. University of Padua, Italy 4. University of Duisburg-Essen, Germany
Abstract
This report documents the program and the outcomes of Dagstuhl Seminar 23031 "Frontiers of Information Access Experimentation for Research and Education", which brought together 38 participants from 12 countries. The seminar addressed technology-enhanced information access (information retrieval, recommender systems, natural language processing) and specifically focused on developing more responsible experimental practices leading to more valid results, both for research as well as for scientific education.
The seminar featured a series of long and short talks delivered by participants, who helped in setting a common ground and in letting emerge topics of interest to be explored as the main output of the seminar. This led to the definition of five groups which investigated challenges, opportunities, and next steps in the following areas:
reality check, i.e. conducting real-world studies, human-machine-collaborative relevance judgment frameworks, overcoming methodological challenges in information retrieval and recommender systems through awareness and education, results-blind reviewing, and guidance for authors.
Date:
15--20 January 2023.
Website:
https://www.dagstuhl.de/23031.
Publisher
Association for Computing Machinery (ACM)
Subject
Hardware and Architecture,Management Information Systems
Reference63 articles.
1. The Impact of Search Engine Selection and Sorting Criteria on Vaccination Beliefs and Attitudes: Two Experiments Manipulating Google Output 2. Timothy G. Armstrong , Alistair Moffat , William Webber , and Justin Zobel . Improvements that don't add up: Ad-hoc retrieval results since 1998 . In Proceedings of the 18th ACM Conference on Information and Knowledge Management, CIKM '09 , pages 601 -- 610 , 2009 . 10.1145/1645953.1646031 Timothy G. Armstrong, Alistair Moffat, William Webber, and Justin Zobel. Improvements that don't add up: Ad-hoc retrieval results since 1998. In Proceedings of the 18th ACM Conference on Information and Knowledge Management, CIKM '09, pages 601--610, 2009. 10.1145/1645953.1646031 3. Christine Bauer and Eva Zangerle . Leveraging multi-method evaluation for multi-stakeholder settings . In 1st Workshop on the Impact of Recommender Systems, co-located with 13th ACM Conference on Recommender Systems (ACM RecSys 2019 ), ImpactRS'19. CEUR-WS.org , 2019 . URL http://ceur-ws.org/Vol-2462/short3.pdf. Christine Bauer and Eva Zangerle. Leveraging multi-method evaluation for multi-stakeholder settings. In 1st Workshop on the Impact of Recommender Systems, co-located with 13th ACM Conference on Recommender Systems (ACM RecSys 2019), ImpactRS'19. CEUR-WS.org, 2019. URL http://ceur-ws.org/Vol-2462/short3.pdf. 4. Christine Bauer , Ben A. Carterette , Nicola Ferro , and Norbert Fuhr , editors. Report from Dagstuhl Seminar 23031: Frontiers of Information Access Experimentation for Research and Education , Dagstuhl Reports , Volume 13 , Number 1 , 2023 a. Schloss Dagstuhl-Leibniz-Zentrum für Informatik, Germany. Christine Bauer, Ben A. Carterette, Nicola Ferro, and Norbert Fuhr, editors. Report from Dagstuhl Seminar 23031: Frontiers of Information Access Experimentation for Research and Education, Dagstuhl Reports, Volume 13, Number 1, 2023a. Schloss Dagstuhl-Leibniz-Zentrum für Informatik, Germany. 5. Christine Bauer , Maik Fröbe , Dietmar Jannach , Udo Kruschwitz , Paolo Rosso , Damiano Spina , and Nava Tintarev . Overcoming methodological challenges in information retrieval and recommender systems through awareness and education . In Bauer et al. [2023a] , pages 51 -- 67 . Christine Bauer, Maik Fröbe, Dietmar Jannach, Udo Kruschwitz, Paolo Rosso, Damiano Spina, and Nava Tintarev. Overcoming methodological challenges in information retrieval and recommender systems through awareness and education. In Bauer et al. [2023a], pages 51--67.
|
|