Stochastic Gradient MCMC for Massive Geostatistical Data: Competing methods and metrics

Table of Links

Abstract and 1 Introduction

1.1 Methods to handle large spatial datasets

1.2 Review of stochastic gradient methods

2 Matern Gaussian Process Model and its Approximations

2.1 The Vecchia approximation

3 The SG-MCMC Algorithm and 3.1 SG Langevin Dynamics

3.2 Derivation of gradients and Fisher information for SGRLD

4 Simulation Study and 4.1 Data generation

4.2 Competing methods and metrics

4.3 Results

5 Analysis of Global Ocean Temperature Data

6 Discussion, Acknowledgements, and References

Appendix A.1: Computational Details

Appendix A.2: Additional Results

4.2 Competing methods and metrics

We compare our SGRLD method with four different MCMC methods. The first three are SG methods with adaptive drifts. The last method uses the full dataset to sample the posterior distribution using the Vecchia approximation. The three SGMCMC methods all use momentum and past gradient information to estimate the curvature and accelerate the convergence. These methods extend the momentum methods used in SG optimization methods for faster exploration of the posterior. The first method is Preconditioned SGLD (pSGLD) of Li et al. (2016a) that uses the Root Mean Square Propagation (RMSPROP) (Hinton et al., 2012) algorithm to estimate a diagonal preconditioner for the minibatch gradient and injected noise. The second method is ADAMSGLD (Kim et al., 2022) that extends the widely used ADAM optimizer (Kingma and Ba, 2014) to the SGLD setting. ADAMSGLD approximates the first-order and second-order moments of the minibatch gradients to construct a preconditioner. Finally, we also include the performance of Momentum SGLD (MSGLD) where no preconditioner is used but past gradient information is used to accelerate the exploration of the posterior. The details of the above algorithms are included in the Appendix A.1. The final method we consider is the Nearest Neighbor Gaussian Process (NNGP) method (Datta et al., 2016). This method is the standard MCMC method based on the Vecchia approximation and is implemented in the R package spNNGP (Finley et al., 2022). For this method, the initial values are set to the true values and the Metropolis-Hastings proposal distribution is chosen adaptively using the default settings.

The prior 90% credible intervals for ρ and ν are (2.06, 7.88) and (0.52, 14.08) respectively, which represent weakly informative priors.