Smooth Factor Analysis (SFA) to Effectively Remove High Levels of Noise from Spectral Data Sets

Abstract

Smooth factor analysis (SFA) is introduced as an effective method of removing heavy noise from spectral data sets. A modified form of the nonlinear iterative partial least squares (NIPALS) algorithm involving the smoothing of factors at each step is used in SFA. Compared with the conventional smoothing techniques for individual spectra, SFA is much more effective in the treatment of very noisy spectra (∼40% noise level). Smooth factor analysis invokes a large number of smooth factors to retain pertinent spectral information for high fidelity without distortion. This approach can be used as an effective general pretreatment procedure for multivariate spectral data analysis, such as principal component analysis (PCA) and partial least squares (PLS). This SFA method was also applied to the real experimental data, and its results successfully demonstrated the powerful potential for effective noise removal. Furthermore, this treatment is found to be very helpful to assist effective interpretation of two-dimensional correlation spectroscopy (2D-COS) spectra with very high noise level, which was not possible before.