Affiliation:
1. City University of Hong Kong, China
Abstract
This article conducts user evaluation to study the performance difference between interactive and automatic search. Particularly, the study aims to provide empirical insights of how the performance landscape of video search changes, with tens of thousands of concept detectors freely available to exploit for query formulation. We compare three types of search modes: free-to-play (i.e., search from scratch), non-free-to-play (i.e., search by inspecting results provided by automatic search), and automatic search including concept-free and concept-based retrieval paradigms. The study involves a total of 40 participants; each performs interactive search over 15 queries of various difficulty levels using two search modes on the IACC.3 dataset provided by TRECVid organizers. The study suggests that the performance of automatic search is still far behind interactive search. Furthermore, providing users with the result of automatic search for exploration does not show obvious advantage over asking users to search from scratch. The study also analyzes user behavior to reveal insights of how users compose queries, browse results, and discover new query terms for search, which can serve as guideline for future research of both interactive and automatic search.
Funder
Research Grants Council of the Hong Kong Special Administrative Region, China
Publisher
Association for Computing Machinery (ACM)
Subject
Computer Networks and Communications,Hardware and Architecture
Reference74 articles.
1. George Awad Asad A. Butt Jonathan G. Fiscus David Joy Andrew Delgado Martial Michel Alan F. Smeaton Yvette Graham Gareth J. F. Jones Wessel Kraaij Georges Quénot Maria Eskevich Roeland Ordelman and Benoit Huet (Eds.). 2017. 2017 TREC Video Retrieval Evaluation (TRECVID’17). National Institute of Standards and Technology (NIST). Retrieved from https://www-nlpir.nist.gov/projects/tvpubs/tv.pubs.17.org.html. George Awad Asad A. Butt Jonathan G. Fiscus David Joy Andrew Delgado Martial Michel Alan F. Smeaton Yvette Graham Gareth J. F. Jones Wessel Kraaij Georges Quénot Maria Eskevich Roeland Ordelman and Benoit Huet (Eds.). 2017. 2017 TREC Video Retrieval Evaluation (TRECVID’17). National Institute of Standards and Technology (NIST). Retrieved from https://www-nlpir.nist.gov/projects/tvpubs/tv.pubs.17.org.html.
Cited by
7 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献