The role of optimization in some recent advances in data-driven decision-making

Abstract

AbstractData-driven decision-making has garnered growing interest as a result of the increasing availability of data in recent years. With that growth many opportunities and challenges have sprung up in the areas of predictive and prescriptive analytics. Often, optimization can play an important role in tackling these issues. In this paper, we review some recent advances that highlight the difference that optimization can make in data-driven decision-making. We discuss some of our contributions that aim to advance both predictive and prescriptive models. First, we describe how we can optimally estimate clustered models that result in improved predictions. Next, we consider how we can optimize over objective functions that arise from tree ensemble models in order to obtain better prescriptions. Finally, we discuss how we can learn optimal solutions directly from the data allowing for prescriptions without the need for predictions. For all these new methods, we stress the need for good performance but also the scalability to large heterogeneous datasets.