Causal Analysis to Explain the Performance of Algorithms: A Case Study for the Bin Packing Problem

Abstract

This work presents a knowledge discovery approach through Causal Bayesian Networks for understanding the conditions under which the performance of an optimization algorithm can be affected by the characteristics of the instances of a combinatorial optimization problem (COP). We introduce a case study for the causal analysis of the performance of two state-of-the-art algorithms for the one-dimensional Bin Packing Problem (BPP). We meticulously selected the set of features associated with the parameters that define the instances of the problem. Subsequently, we evaluated the algorithmic performance on instances with distinct features. Our analysis scrutinizes both instance features and algorithm performance, aiming to identify causes influencing the performance of the algorithms. The proposed study successfully identifies specific values affecting algorithmic effectiveness and efficiency, revealing shared causes within some value ranges across both algorithms. The knowledge generated establishes a robust foundation for future research, enabling predictions of algorithmic performance, as well as the selection and design of heuristic strategies for improving the performance in the most difficult instances. The causal analysis employed in this study did not require specific configurations, making it an invaluable tool for analyzing the performance of different algorithms in other COPs.