Fast Stochastic Nearest Neighbor Pairwise Composite Likelihood for Massive Spatial Datasets

2026-07-07

AI Summary

This work addresses the computational challenge of nearest-neighbor-based pairwise composite likelihood estimation in large-scale spatial data, where balancing statistical efficiency and scalability is difficult. The authors propose an efficient estimation framework that first constructs a deterministic nearest-neighbor candidate graph and then applies two randomized sparsification strategies—Bernoulli sparsification and fixed-budget sparsification—to subsample pairwise likelihood contributions. This approach substantially reduces computational cost while preserving estimation accuracy for key covariance parameters, including the mean, range, and sill. Experiments on a dataset of 2.5 million temperature observations demonstrate a significant acceleration in covariance fitting, with predictive performance comparable to that of the Vecchia approximation, thereby confirming both the scalability and statistical validity of the proposed method.

Abstract

Weighted pairwise composite likelihoods based on nearest-neighbor (NN) pairs provide a scalable alternative to full likelihood inference for spatial random fields, but can remain expensive when moderately large NN neighborhoods are needed. We propose a stochastic acceleration that constructs the deterministic NN candidate graph and evaluates only a randomized subset of its pairwise contributions. We consider two thinning designs: Bernoulli thinning, which controls the retained-pair budget in expectation, and fixed-budget thinning, which enforces an exact budget through target-wise sampling without replacement. Simulation studies for Matérn covariance models suggest that, in the settings considered, retaining two pairs per observation provides a stable statistical--computational compromise. The stochastic NN pairwise estimators provide a faster alternative to a Vecchia-type Gaussian approximation when substantial reductions in covariance-fitting time are desired and a modest loss of efficiency is acceptable. This trade-off is especially favorable for the mean, scale, and sill parameters, while the main efficiency loss is concentrated on smoothness estimation. In an application to July average temperature over the western--central United States, based on 2.5 million WorldClim observations, the proposed estimators achieve predictive accuracy essentially indistinguishable from the Vecchia benchmark, with substantially shorter covariance-fitting time.

Problem

• spatial data
• nearest-neighbor
• composite likelihood
• scalability
• computational cost

Innovation

• stochastic nearest neighbor
• pairwise composite likelihood
• scalable spatial inference
• covariance estimation
• Vecchia approximation