1. Shallow pooling for sparse labels
2. Javed A. Aslam and Robert Savell. 2003. On the effectiveness of evaluating retrieval systems in the absence of relevance judgments . In SIGIR 2003: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval , July 28 - August 1, 2003 , Toronto, Canada, Charles L. A. Clarke, Gordon V. Cormack, Jamie Callan, David Hawking, and Alan F. Smeaton (Eds.). ACM, 361--362. https://doi.org/10.1145/860435.860501 10.1145/860435.860501 Javed A. Aslam and Robert Savell. 2003. On the effectiveness of evaluating retrieval systems in the absence of relevance judgments. In SIGIR 2003: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, July 28 - August 1, 2003, Toronto, Canada, Charles L. A. Clarke, Gordon V. Cormack, Jamie Callan, David Hawking, and Alan F. Smeaton (Eds.). ACM, 361--362. https://doi.org/10.1145/860435.860501
3. Deciding on an adjustment for multiplicity in IR experiments
4. Chris Buckley and Ellen M. Voorhees . 2004. Retrieval evaluation with incomplete information . In SIGIR 2004: Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval , Sheffield, UK, July 25--29 , 2004 , Mark Sanderson, Kalervo J"a rvelin, James Allan, and Peter Bruza (Eds.). ACM, 25--32. https://doi.org/10.1145/1008992.1009000 10.1145/1008992.1009000 Chris Buckley and Ellen M. Voorhees. 2004. Retrieval evaluation with incomplete information. In SIGIR 2004: Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Sheffield, UK, July 25--29, 2004, Mark Sanderson, Kalervo J"a rvelin, James Allan, and Peter Bruza (Eds.). ACM, 25--32. https://doi.org/10.1145/1008992.1009000
5. Stefan Bü ttcher, Charles L. A. Clarke , Peter C. K. Yeung , and Ian Soboroff . 2007 . Reliable information retrieval evaluation with incomplete and biased judgements . In SIGIR 2007: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval , Amsterdam, The Netherlands, July 23--27 , 2007, Wessel Kraaij, Arjen P. de Vries, Charles L. A. Clarke, Norbert Fuhr, and Noriko Kando (Eds.). ACM, 63--70. https://doi.org/10.1145/1277741.1277755 10.1145/1277741.1277755 Stefan Bü ttcher, Charles L. A. Clarke, Peter C. K. Yeung, and Ian Soboroff. 2007. Reliable information retrieval evaluation with incomplete and biased judgements. In SIGIR 2007: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Amsterdam, The Netherlands, July 23--27, 2007, Wessel Kraaij, Arjen P. de Vries, Charles L. A. Clarke, Norbert Fuhr, and Noriko Kando (Eds.). ACM, 63--70. https://doi.org/10.1145/1277741.1277755