SIGIR Initiative to Implement ACM Artifact Review and Badging

Abstract

Recently, the ACM created a policy on Artifact Review and Badging, which presents a framework to help SIGs recognize repeatability, replicability and reproducibility in published research. While the ACM policy established a vocabulary and definitions, it did not prescribe procedures for implementation. Rather, the ACM has left this to each SIG to define given the variety of research traditions and approaches that exist with the ACM community. SIGs are not required to implement badging, but given the growing interest in this topic in the SIGIR community, a task force has been assembled to determine how badging might be implemented. This report describes the ACM policy on Artifact Review and Badging, introduces the task force, and presents survey data describing the SIGIR community's opinions about this initiative.