Iterative Rule Extension for Logic Analysis of Data: An MILP-Based Heuristic to Derive Interpretable Binary Classifiers from Large Data Sets

Abstract

Data-driven decision making is rapidly gaining popularity, fueled by the ever-increasing amounts of available data and encouraged by the development of models that can identify nonlinear input–output relationships. Simultaneously, the need for interpretable prediction and classification methods is increasing as this improves both our trust in these models and the amount of information we can abstract from data. An important aspect of this interpretability is to obtain insight in the sensitivity–specificity trade-off constituted by multiple plausible input–output relationships. These are often shown in a receiver operating characteristic curve. These developments combined lead to the need for a method that can identify complex yet interpretable input–output relationships from large data, that is, data containing large numbers of samples and features. Boolean phrases in disjunctive normal form (DNF) are highly suitable for explaining nonlinear input–output relationships in a comprehensible way. Mixed integer linear programming can be used to obtain these Boolean phrases from binary data though its computational complexity prohibits the analysis of large data sets. This work presents IRELAND, an algorithm that allows for abstracting Boolean phrases in DNF from data with up to 10,000 samples and features. The results show that, for large data sets, IRELAND outperforms the current state of the art in terms of prediction accuracy. Additionally, by construction, IRELAND allows for an efficient computation of the sensitivity–specificity trade-off curve, allowing for further understanding of the underlying input–output relationship. History: Accepted by Andrea Lodi, Area Editor for Design & Analysis of Algorithms–Discrete. Funding: This work was supported by the Netherlands Organization for Scientific Research (Nederlandse Organisatie voor Wetenschappelijk Onderzoek) [Grant VI.VENI.192.043]. Supplemental Material: The online supplement is available at https://doi.org/10.1287/ijoc.2021.0284 .