K0_fn {ZVCV}

Kernel matrix calculation

Description

This function calculates the full K_0 matrix, which is a first or second order Stein operator applied to a standard kernel. The output of this function can be used as an argument to CF, CF_crossval, SECF, SECF_crossval, aSECF and aSECF_crossval. The kernel matrix is automatically computed in all of the above methods, but it is faster to calculate in advance when using more than one of the above functions and when using any of the crossval functions.

Usage

K0_fn(
  samples,
  derivatives,
  sigma,
  steinOrder,
  kernel_function,
  Z = NULL,
  nystrom_inds = NULL
)

Arguments

samples

An N by d matrix of samples from the target

derivatives

An N by d matrix of derivatives of the log target with respect to the parameters

sigma

The tuning parameters of the specified kernel. This involves a single length-scale parameter in "gaussian" and "RQ", a length-scale and a smoothness parameter in "matern" and two parameters in "product" and "prodsim". See below for further details.

steinOrder

This is the order of the Stein operator. The default is 1 in the control functionals paper (Oates et al, 2017) and 2 in the semi-exact control functionals paper (South et al, 2020). The following values are currently available: 1 for all kernels and 2 for "gaussian", "matern" and "RQ". See below for further details.

kernel_function

Choose between "gaussian", "matern", "RQ", "product" or "prodsim". See below for further details.

Z

(optional) An N by N (or N by m where m is the length of nystrom_inds). This can be calculated using squareNorm.

nystrom_inds

(optional) The sample indices to be used in the Nystrom approximation (for when using aSECF).

Value

An N by N kernel matrix (or N by m where m is the length of nystrom_inds).

On the choice of \sigma, the kernel and the Stein order

The kernel in Stein-based kernel methods is L_x L_y k(x,y) where L_x is a first or second order Stein operator in x and k(x,y) is some generic kernel to be specified.

The Stein operators for distribution p(x) are defined as:

• steinOrder=1: L_x g(x) = \nabla_x^T g(x) + \nabla_x \log p(x)^T g(x) (see e.g. Oates el al (2017))

• steinOrder=2: L_x g(x) = \Delta_x g(x) + \nabla_x log p(x)^T \nabla_x g(x) (see e.g. South el al (2020))

Here \nabla_x is the first order derivative wrt x and \Delta_x = \nabla_x^T \nabla_x is the Laplacian operator.

The generic kernels which are implemented in this package are listed below. Note that the input parameter sigma defines the kernel parameters \sigma.

• "gaussian": A Gaussian kernel,

  k(x,y) = exp(-z(x,y)/\sigma^2)

• "matern": A Matern kernel with \sigma = (\lambda,\nu),

  k(x,y) = bc^{\nu}z(x,y)^{\nu/2}K_{\nu}(c z(x,y)^{0.5})

  where b=2^{1-\nu}(\Gamma(\nu))^{-1}, c=(2\nu)^{0.5}\lambda^{-1} and K_{\nu}(x) is the modified Bessel function of the second kind. Note that \lambda is the length-scale parameter and \nu is the smoothness parameter (which defaults to 2.5 for steinOrder=1 and 4.5 for steinOrder=2).

• "RQ": A rational quadratic kernel,

  k(x,y) = (1+\sigma^{-2}z(x,y))^{-1}

• "product": The product kernel that appears in Oates et al (2017) with \sigma = (a,b)

  k(x,y) = (1+a z(x) + a z(y))^{-1} exp(-0.5 b^{-2} z(x,y))

• "prodsim": A slightly different product kernel with \sigma = (a,b) (see e.g. https://www.imperial.ac.uk/inference-group/projects/monte-carlo-methods/control-functionals/),

  k(x,y) = (1+a z(x))^{-1}(1 + a z(y))^{-1} exp(-0.5 b^{-2} z(x,y))

In the above equations, z(x) = \sum_j x[j]^2 and z(x,y) = \sum_j (x[j] - y[j])^2. For the last two kernels, the code only has implementations for steinOrder=1. Each combination of steinOrder and kernel_function above is currently hard-coded but it may be possible to extend this to other kernels in future versions using autodiff. The calculations for the first three kernels above are detailed in South et al (2020).

Author(s)

Leah F. South

References

Oates, C. J., Girolami, M. & Chopin, N. (2017). Control functionals for Monte Carlo integration. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 79(3), 695-718.

South, L. F., Karvonen, T., Nemeth, C., Girolami, M. and Oates, C. J. (2020). Semi-Exact Control Functionals From Sard's Method. https://arxiv.org/abs/2002.00033