Component-based regularization of a multivariate GLM with a thematic partitioning of the explanatory variables

Abstract

We address component-based regularization of a multivariate generalized linear model (GLM). A vector of random responses [Formula: see text] is assumed to depend, through a GLM, on a set [Formula: see text] of explanatory variables, as well as on a set [Formula: see text] of additional covariates. [Formula: see text] is partitioned into [Formula: see text] conceptually homogenous variable groups [Formula: see text], viewed as explanatory themes. Variables in each [Formula: see text] are assumed many and redundant. Thus, generalized linear regression demands dimension reduction and regularization with respect to each [Formula: see text]. By contrast, variables in [Formula: see text] are assumed few and selected so as to demand no regularization. Regularization is performed searching each [Formula: see text] for an appropriate number of orthogonal components that both contribute to model [Formula: see text] and capture relevant structural information in [Formula: see text]. To estimate a single-theme model, we first propose an enhanced version of Supervised Component Generalized Linear Regression (SCGLR), based on a flexible measure of structural relevance of components, and able to deal with mixed-type explanatory variables. Then, to estimate the multiple-theme model, we develop an algorithm encapsulating this enhanced SCGLR: THEME-SCGLR. The method is tested on simulated data and then applied to rainforest data in order to model the abundance of tree species.