Simultaneous Dimension Reduction and Variable Selection for Multinomial Logistic Regression

Abstract

Multinomial logistic regression is a useful model for predicting the probabilities of multiclass outcomes. Because of the complexity and high dimensionality of some data, it is challenging to fit a valid model with high accuracy and interpretability. We propose a novel sparse reduced-rank multinomial logistic regression model to jointly select variables and reduce the dimension via a nonconvex row constraint. We develop a block-wise iterative algorithm with a majorizing surrogate function to efficiently solve the optimization problem. From an algorithmic aspect, we show that the output estimator enjoys consistency in estimation and sparsity recovery even in a high-dimensional setting. The finite sample performance of the proposed method is investigated via simulation studies and two real image data sets. The results show that our proposal has competitive performance in both estimation accuracy and computation time. History: Accepted by Andrea Lodi, Area Editor for Design & Analysis of Algorithms–Discrete. Funding: This work was supported by the National Natural Science Foundation of China [Grants 71991474, 12171449, 11801540, and 12071494] and the Natural Science Foundation of Anhui Province [Grant BJ2040170017]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/ijoc.2022.0132 .