1. 1,500 scientists lift the lid on reproducibility
2. Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references
3. Xavier Bouthillier Pierre Delaunay Mirko Bronzi Assya Trofimov Brennan Nichyporuk Justin Szeto Naz Sepah Edward Raff Kanika Madan Vikram Voleti Samira Ebrahimi Kahou Vincent Michalski Dmitriy Serdyuk Tal Arbel Chris Pal Gaël Varoquaux and Pascal Vincent. 2021. Accounting for Variance in Machine Learning Benchmarks. In Machine Learning and Systems (MLSys). http://arxiv.org/abs/2103.03098 Xavier Bouthillier Pierre Delaunay Mirko Bronzi Assya Trofimov Brennan Nichyporuk Justin Szeto Naz Sepah Edward Raff Kanika Madan Vikram Voleti Samira Ebrahimi Kahou Vincent Michalski Dmitriy Serdyuk Tal Arbel Chris Pal Gaël Varoquaux and Pascal Vincent. 2021. Accounting for Variance in Machine Learning Benchmarks. In Machine Learning and Systems (MLSys). http://arxiv.org/abs/2103.03098
4. Xavier Bouthillier , César Laurent , and Pascal Vincent . 2019 . Unreproducible Research is Reproducible . In Proceedings of the 36th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 97) , Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, Long Beach, California, USA, 725–734. http://proceedings.mlr.press/v97/bouthillier19a.html Xavier Bouthillier, César Laurent, and Pascal Vincent. 2019. Unreproducible Research is Reproducible. In Proceedings of the 36th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, Long Beach, California, USA, 725–734. http://proceedings.mlr.press/v97/bouthillier19a.html
5. Benjamin Callahan , Diana Proctor , David Relman , Julia Fukuyama , and Susan Holmes . 2016 . REPRODUCIBLE RESEARCH WORKFLOW IN R FOR THE ANALYSIS OF PERSONALIZED HUMAN MICROBIOME DATA.Pacific Symposium on Biocomputing. Pacific Symposium on Biocomputing 21 (2016), 183–94. http://www.ncbi.nlm.nih.gov/pubmed/26776185http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC4873301 Benjamin Callahan, Diana Proctor, David Relman, Julia Fukuyama, and Susan Holmes. 2016. REPRODUCIBLE RESEARCH WORKFLOW IN R FOR THE ANALYSIS OF PERSONALIZED HUMAN MICROBIOME DATA.Pacific Symposium on Biocomputing. Pacific Symposium on Biocomputing 21 (2016), 183–94. http://www.ncbi.nlm.nih.gov/pubmed/26776185http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC4873301