PROMISE retreat report prospects and opportunities for information access evaluation-Reference-Cited by-同舟云学术

PROMISE retreat report prospects and opportunities for information access evaluation

Published:2012-12-21 Issue:2 Volume:46 Page:60-84
ISSN:0163-5840
Container-title:ACM SIGIR Forum
language:en
Short-container-title:SIGIR Forum

Author:

Agosti Maristella¹,Berendsen Richard²,Bogers Toine³,Braschler Martin⁴,Buitelaar Paul⁵,Choukri Khalid⁶,Maria Di Nunzio Giorgio¹,Ferro Nicola¹,Forner Pamela⁷,Hanbury Allan⁸,Heppin Karin Friberg⁹,Hansen Preben¹⁰,Järvelin Anni¹⁰,Larsen Birger³,Lupu Mihai⁸,Masiero Ivano¹,Müller Henning¹¹,Peruzzo Simone¹,Petras Vivien¹²,Piroi Florina⁸,de Rijke Maarten²,Santucci Giuseppe¹³,Silvello Gianmaria¹,Toms Elaine¹⁴

Affiliation:

1. University of Padua, Italy

2. University of Amsterdam, The Netherlands

3. Royal School of Library and Information Science, Denmark

4. Zurich University of Applied Sciences, Switzerland

5. National University of Ireland, Galway Ireland

6. Evaluations and Language resources Distribution Agency (ELDA), France

7. Centre for the Evaluation of Language and Communication Technologies (CELCT), Italy

8. Vienna University of Technology, Austria

9. University of Gothenburg, Sweden

10. Swedish Institute of Computer Science, Sweden

11. University of Applied Sciences Western Switzerland (HES-SO), Switzerland

12. Humboldt University Berlin, Germany

13. Sapienza, University of Rome, Italy

14. University of Sheffield, United Kingdom

Abstract

The PROMISE network of excellence organized a two-days brainstorming workshop on 30th and 31st May 2012 in Padua, Italy, to discuss and envisage future directions and perspectives for the evaluation of information access and retrieval systems in multiple languages and multiple media. This document reports on the outcomes of this event and provides details about the six envisaged research lines: search applications; contextual evaluation; challenges in test collection design and exploitation; component-based evaluation; ongoing evaluation; and signal-aware evaluation. The ultimate goal of the PROMISE retreat is to stimulate and involve the research community along these research lines and to provide funding agencies with effective and scientifically sound ideas for coordinating and supporting information access research.

Publisher

Association for Computing Machinery (ACM)

Subject

Hardware and Architecture,Management Information Systems

Link

https://dl.acm.org/doi/pdf/10.1145/2422256.2422265

Reference57 articles.

1. M. Agosti R. Berendsen T. Bogers M. Braschler P. Buitelaar K. Choukri G. M. Di Nunzio N. Ferro P. Forner A. Hanbury K. Friberg Heppin P. Hansen A. Järvelin B. Larsen M. Lupu I. Masiero H. Müller S. Peruzzo V. Petras F. Piroi M. de Rijke G. Santucci G. Silvello and E. Toms. PROMISE Retreat Report -- Prospects and Opportunities for Information Access Evaluation. PROMISE network of excellence ISBN 978-88-6321-039-2 http://www.promise-noe.eu/promise-retreat-report-2012/ September 2012 M. Agosti R. Berendsen T. Bogers M. Braschler P. Buitelaar K. Choukri G. M. Di Nunzio N. Ferro P. Forner A. Hanbury K. Friberg Heppin P. Hansen A. Järvelin B. Larsen M. Lupu I. Masiero H. Müller S. Peruzzo V. Petras F. Piroi M. de Rijke G. Santucci G. Silvello and E. Toms. PROMISE Retreat Report -- Prospects and Opportunities for Information Access Evaluation. PROMISE network of excellence ISBN 978-88-6321-039-2 http://www.promise-noe.eu/promise-retreat-report-2012/ September 2012

2. M. Agosti M. Braschler E. Di Buccio M. Dussin N. Ferro G. L. Granato I. Masiero E. Pianta G. Santucci G. Silvello and G. Tino. Deliverable D3.2 -- Specification of the evaluation infrastructure based on user requirements. PROMISE Network of Excellence EU 7FP Contract N. 258191. http://www.promise-noe.eu/documents/10156/fdf43394-0997-4638-9f99-38b2e9c63802 August 2011. M. Agosti M. Braschler E. Di Buccio M. Dussin N. Ferro G. L. Granato I. Masiero E. Pianta G. Santucci G. Silvello and G. Tino. Deliverable D3.2 -- Specification of the evaluation infrastructure based on user requirements. PROMISE Network of Excellence EU 7FP Contract N. 258191. http://www.promise-noe.eu/documents/10156/fdf43394-0997-4638-9f99-38b2e9c63802 August 2011.

3. M. Agosti E. Di Buccio N. Ferro I. Masiero M. Nicchio S. Peruzzo and G. Silvello. Deliverable D3.3 -- Prototype of the Evaluation Infrastructure. PROMISE Network of Excellence EU 7FP Contract N. 258191. http://www.promise-noe.eu/documents/10156/3783730a-bce3-481b-83df-48e209c6286a September 2012. M. Agosti E. Di Buccio N. Ferro I. Masiero M. Nicchio S. Peruzzo and G. Silvello. Deliverable D3.3 -- Prototype of the Evaluation Infrastructure. PROMISE Network of Excellence EU 7FP Contract N. 258191. http://www.promise-noe.eu/documents/10156/3783730a-bce3-481b-83df-48e209c6286a September 2012.

4. M. Agosti E. Di Buccio N. Ferro I. Masiero S. Peruzzo and G. Silvello. DIRECTions: Design and Specication of an IR Evaluation Infrastructure. In Catarci et al. {24}. 10.1007/978-3-642-33247-0_11 M. Agosti E. Di Buccio N. Ferro I. Masiero S. Peruzzo and G. Silvello. DIRECTions: Design and Specication of an IR Evaluation Infrastructure. In Catarci et al. {24}. 10.1007/978-3-642-33247-0_11

Cited by 8 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. A General Linear Mixed Models Approach to Study System Component Effects;Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval;2016-07-07

2. Semantic representation and enrichment of information retrieval experimental data;International Journal on Digital Libraries;2016-05-28

3. Keyword-Based Search Over Databases: A Roadmap for a Reference Architecture Paired with an Evaluation Framework;Lecture Notes in Computer Science;2016

4. Visual Analytics for Information Retrieval Evaluation (VAIRË 2015);Lecture Notes in Computer Science;2015

5. CLEF 15th Birthday;ACM SIGIR Forum;2014-12-23