Augmenting Stationary Covariance Functions with a Smoothness Hyperparameter and Improving Gaussian Process Regression Using a Structural Similarity Index

Abstract

Abstract Gaussian process (GP) regression provides a probabilistic framework for modeling geochemistry in mineral resource estimation and environmental monitoring applications. An issue with this approach is that the kernel hyperparameters obtained by maximizing the log-marginal likelihood (LML) often produce GP posterior mean estimates that are overly smooth. This motivates the development of augmented kernels that are more capable of capturing the variability inherent in geological/geochemical processes than the existing stationary covariance functions like the Matérn kernels. This paper makes two contributions. First, it describes an extended class of stationary kernels that contain an extra smoothness hyperparameter ($$\alpha $$ α ) which can be learned from the input data. Valid intervals for $$\alpha $$ α that lead to positive semi-definiteness are determined. Second, it uses a statistical measure called the structural similarity index (SSIM) to quantify smoothness and the spatial fidelity of GP solutions with respect to the input samples. This provides a new way for validating and optimizing GP models. Statistical and spectral analyses provide insights into the behavior of $$\alpha $$ α in the augmented kernels which retain useful properties such as sparsity. Results on the Northern Great Basin geochemical dataset demonstrate that, all things being equal, (1) adjusting $$\alpha $$ α increases spatial fidelity in GP regression; (2) SSIM is a more reliable spatial quality measure than LML; and (3) the optimal $$\alpha $$ α value obtained is correlated with the Wiener entropy of the random process, which indicates the spectral flatness of the chemical signal in the Fourier domain. For GP regression, function smoothness may be defined using Sobolev space. The results show that $$\alpha $$ α regulates over-smoothing by moderating the rate of decay in the power spectrum of the equivalent kernel. Graphic abstract