Sparse Convex Regression

Abstract

We consider the problem of best [Formula: see text]-subset convex regression using [Formula: see text] observations in [Formula: see text] variables. For the case without sparsity, we develop a scalable algorithm for obtaining high quality solutions in practical times that compare favorably with other state of the art methods. We show that by using a cutting plane method, the least squares convex regression problem can be solved for sizes [Formula: see text] in minutes and [Formula: see text] in hours. Our algorithm can be adapted to solve variants such as finding the best convex or concave functions with coordinate-wise monotonicity, norm-bounded subgradients, and minimize the [Formula: see text] loss—all with similar scalability to the least squares convex regression problem. Under sparsity, we propose algorithms which iteratively solve for the best subset of features based on first order and cutting plane methods. We show that our methods scale for sizes [Formula: see text] in minutes and [Formula: see text] in hours. We demonstrate that these methods control for the false discovery rate effectively.