1. Armstrong, T.G., Moffat, A., Webber, W., Zobel, J.: Improvements that don’t add up: ad-hoc retrieval results since 1998. In: Proceedings of CIKM, pp. 601–610 (2009)
2. Baker, M.: 1,500 scientists lift the lid on reproducibility. Nature 533, 452–454 (2016)
3. Balog, K., Kelly, L., Schuth, A.: Head first: living labs for ad-hoc search evaluation. In: Proceedings of CIKM, pp. 1815–1818 (2014)
4. Breuer, T., Schaer, P.: Dockerizing automatic routing runs for the open-source IR replicability challenge (osirrc 2019). In: Proceedings of the Open-Source IR Replicability Challenge (OSIRRC) @ SIGIR (2019)
5. Breuer, T., Schaer, P.: Replicability and reproducibility of automatic routing runs. In: Working Notes of CLEF. CEUR Workshop Proceedings (2019)