Novel Non-Convex Regularization for Generating Double Threshold Value in Penalized Least Squares Regression

Abstract

A lot of works in machine learning (ML) usually use components which are built by signal processing, such as wavelet and Fourier coefficients, for computations; therefore, many problems in signal processing affect ML algorithms. Consequently, we try to solve linear inverse problems in signal processing, such as the classical denoising and deconvolution problems, in this work. The penalized least squares regression (PLSR), also called as the proximity operator, of the non-convex regularization, can solve various problems in signal processing. In the classical denoising problem, the PLSR with the double threshold value is more flexible than the PLSR with the single threshold value. Therefore, we propose the novel non-convex regularization which can build the PLSR with the double threshold value. The proposed regularization is based on good properties of the minimax-concave (MC) regularization. We also present novel PLSRs of the MC and proposed regularizations where these PLSRs have closed-form solutions and the relationship to the group of considered components, also known as the multivariate case, together. We compare proposed methods with state-of-the-art methods in the classical denoising and deconvolution problems. Here, we use the majorization-minimization (MM) method for comparing the performance of regularizations in the deconvolution problem. Experimental results show that proposed methods give trustworthy results.