Optimal cardinal contests

Abstract

We study the design of crowdsourcing contests in settings where the outputs of the contestants are quantifiable, for example, a data science challenge. This setting is in contrast to those where the output is only qualitative and cannot be objectively quantified, for example, when the goal of the contest is to design a logo. The literature on crowdsourcing contests focuses largely on ordinal contests, where contestants' outputs are ranked by the designer and awards are based on relative ranks. Such contests are ideally suited for the latter setting, where output is qualitative. For our setting (quantitative output), it is possible to design cardinal contests, where awards could be based on the actual outputs and not on their ranking alone—thus, the family of cardinal contests includes the family of ordinal contests. We study the problem of designing an optimal cardinal contest. We use mechanism design theory to derive an optimal cardinal mechanism and provide a convenient implementation—a decreasing reward‐meter mechanism—of the optimal contest. We establish the practicality of our mechanism by showing that it is “Obviously Strategy‐Proof,” a recently introduced formal notion of simplicity in the literature. We also compare the optimal cardinal contest with the most popular ordinal contest—namely, the Winner‐Takes‐All (WTA) contest, along several metrics. In particular, the optimal cardinal mechanism delivers a superior expected best output, whereas the WTA contest yields a greater expected contestant welfare. Furthermore, under a sufficiently large budget, the contest designer's expected net‐benefit is higher under the optimal cardinal mechanism than that under the WTA contest, regardless of the number of contestants in the two mechanisms. Our numerical analysis suggests that, for the contest designer, the average improvement provided by the optimal cardinal mechanism over the WTA contest is about 23%. For a given number of contestants, the benefit of the optimal cardinal mechanism is especially appreciable for projects where the ratio of the designer's utility to agents' cost‐of‐effort falls within a wide practical range. For projects where this ratio is very high, the expected profit of the best WTA contest is reasonably close to that of the optimal cardinal mechanism.