Analyzing market basket data through sparse multivariate logit models

Abstract

AbstractUsing multivariate logit models, we analyze purchases of product categories made by individual households. We introduce a sparse multivariate logit model that considers only a subset of all two-way interactions. A combined forward and backward selection procedure based on a cross-validated performance measure excludes about 74 % of the possible two-way interactions. We also specify random coefficient versions of both the non-sparse and the sparse model. The fact that the random coefficient models lead to better values of the Bayesian information criterion demonstrates the importance of latent heterogeneity. The random coefficients sparse model attains the best statistical performance if we consider model complexity and offers a better interpretability. We investigate the cross-purchase effects of household segments derived from this random coefficient model. As additional interpretation aid we cluster categories and category pairs by integer programming. We demonstrate what the best performing sparse model implies for cross-selling by product recommendations and store layout. The sparse model leads to managerial implications with respect to the effects of advertising in local newspapers and flyers that are as a rule close to those implied by its non-sparse counterpart.