Effective interpretable learning for large-scale categorical data

Abstract

AbstractLarge scale categorical datasets are ubiquitous in machine learning and the success of most deployed machine learning models rely on how effectively the features are engineered. For large-scale datasets, parametric methods are generally used, among which three strategies for feature engineering are quite common. The first strategy focuses on managing the breadth (or width) of a network, e.g., generalized linear models (aka. ). The second strategy focuses on the depth of a network, e.g., Artificial Neural networks or (aka. ). The third strategy relies on factorizing the interaction terms, e.g., Factorization Machines (aka. ). Each of these strategies brings its own advantages and disadvantages. Recently, it has been shown that for categorical data, combination of various strategies leads to excellent results. For example,  -Learning, , etc., leads to state-of-the-art results. Following the trend, in this work, we have proposed another learning framework—-Learning, based on the combination of , , , and a newly introduced component named  (). is in the form of a Bayesian network classifier whose structure is learned apriori, and parameters are learned by optimizing a joint objective function along with , and  parts. We denote the learning of  parameters as . Additionally, the parameters of  are constrained to be actual probabilities—therefore, it is extremely interpretable. Furthermore, one can sample or generate data from , which can facilitate learning and provides a framework for knowledge-guided machine learning. We demonstrate that our proposed framework possesses the resilience to maintain excellent classification performance when confronted with biased datasets. We evaluate the efficacy of our framework in terms of classification performance on various benchmark large-scale categorical datasets and compare against state-of-the-art methods. It is shown that, framework (a) exhibits superior performance on classification tasks, (b) boasts outstanding interpretability and (c) demonstrates exceptional resilience and effectiveness in scenarios involving skewed distributions.