modelKriging {CEGO}

Kriging Model

Description

Implementation of a distance-based Kriging model, e.g., for mixed or combinatorial input spaces. It is based on employing suitable distance measures for the samples in input space.

Usage

modelKriging(x, y, distanceFunction, control = list())

Arguments

x

list of samples in input space

y

column vector of observations for each sample

distanceFunction

a suitable distance function of type f(x1,x2), returning a scalar distance value, preferably between 0 and 1. Maximum distances larger 1 are no problem, but may yield scaling bias when different measures are compared. Should be non-negative and symmetric. It can also be a list of several distance functions. In this case, Maximum Likelihood Estimation (MLE) is used to determine the most suited distance measure. The distance function may have additional parameters. For that case, see distanceParametersLower/Upper in the controls. If distanceFunction is missing, it can also be provided in the control list.

control

(list), with the options for the model building procedure:

lower

lower boundary for theta, default is 1e-6

upper

upper boundary for theta, default is 100

corr

function to be used for correlation modelling, default is fcorrGauss

algTheta

algorithm used to find theta (as well as p and lambda), default is optimInterface.

algThetaControl

list of controls passed to algTheta.

useLambda

whether or not to use the regularization constant lambda (nugget effect). Default is FALSE.

lambdaLower

lower boundary for lambda (log scale), default is -6

lambdaUpper

upper boundary for lambda (log scale), default is 0

distanceParametersLower

lower boundary for parameters of the distance function, default is NA which means there are no distance function parameters. If several distance functions are supplied, this should be a list of lower boundary vectors for each function.

distanceParametersUpper

upper boundary for parameters of the distance function, default is NA which means there are no distance function parameters. If several distance functions are supplied, this should be a list of upper boundary vectors for each function.

distances

a distance matrix. If available, this matrix is used for model building, instead of calculating the distance matrix using the parameters distanceFunction. Default is NULL.

scaling

If TRUE: Distances values are divided by maximum distance to avoid scale bias.

reinterpolate

If TRUE: reinterpolation is used to generate better uncertainty estimates in the presence of noise.

combineDistances

By default, several distance functions or matrices are subject to a likelihood based decision, choosing one. If this parameter is TRUE, they are instead combined by determining a weighted sum. The weighting parameters are determined by MLE.

userParameters

By default: (NULL). Else, this vector is used instead of MLE to specify the model parameters, in the following order: kernel parameters, distance weights, lambda, distance parameters.

indefiniteMethod

The specific method used for correction: spectrum "clip", spectrum "flip", spectrum "square", spectrum "diffusion", feature embedding "feature", nearest definite matrix "near". Default is no correction: "none". See Zaefferer and Bartz-Beielstein (2016).

indefiniteType

The general type of correction for indefiniteness: "NSD","CNSD" or the default "PSD". See Zaefferer and Bartz-Beielstein (2016). Note, that feature embedding may not work in case of multiple distance functions.

indefiniteRepair

boolean, whether conditions of the distance matrix (in case of "NSD","CNSD" correction type) or correlation matrix (in case of "PSD" correction type) are repaired.

Details

The basic Kriging implementation is based on the work of Forrester et al. (2008). For adaptation of Kriging to mixed or combinatorial spaces, as well as choosing distance measures with Maximum Likelihood Estimation, see the other two references (Zaefferer et al., 2014).

Value

an object of class modelKriging containing the options (see control parameter) and determined parameters for the model:

theta

parameters of the kernel / correlation function determined with MLE.

lambda

regularization constant (nugget) lambda

yMu

vector of observations y, minus MLE of mu

SSQ

Maximum Likelihood Estimate (MLE) of model parameter sigma^2

mu

MLE of model parameter mu

Psi

correlation matrix Psi

Psinv

inverse of Psi

nevals

number of Likelihood evaluations during MLE of theta/lambda/p

distanceFunctionIndexMLE

If a list of several distance measures (distanceFunction) was given, this parameter contains the index value of the measure chosen with MLE.

References

Forrester, Alexander I.J.; Sobester, Andras; Keane, Andy J. (2008). Engineering Design via Surrogate Modelling - A Practical Guide. John Wiley & Sons.

Zaefferer, Martin; Stork, Joerg; Friese, Martina; Fischbach, Andreas; Naujoks, Boris; Bartz-Beielstein, Thomas. (2014). Efficient global optimization for combinatorial problems. In Proceedings of the 2014 conference on Genetic and evolutionary computation (GECCO '14). ACM, New York, NY, USA, 871-878. DOI=10.1145/2576768.2598282

Zaefferer, Martin; Stork, Joerg; Bartz-Beielstein, Thomas. (2014). Distance Measures for Permutations in Combinatorial Efficient Global Optimization. In Parallel Problem Solving from Nature - PPSN XIII (p. 373-383). Springer International Publishing.

Zaefferer, Martin and Bartz-Beielstein, Thomas (2016). Efficient Global Optimization with Indefinite Kernels. Parallel Problem Solving from Nature-PPSN XIV. Accepted, in press. Springer.

See Also

predict.modelKriging

Examples

# Set random number generator seed
set.seed(1)
# Simple test landscape
fn <- landscapeGeneratorUNI(1:5,distancePermutationHamming)
# Generate data for training and test
x <- unique(replicate(40,sample(5),FALSE))
xtest <- x[-(1:15)]
x <- x[1:15]
# Determin true objective function values
y <- fn(x)
ytest <- fn(xtest)
# Build model
fit <- modelKriging(x,y,distancePermutationHamming,
    control=list(algThetaControl=list(method="L-BFGS-B"),useLambda=FALSE))
# Predicted obj. function values
ypred <- predict(fit,xtest)$y
# Uncertainty estimate
fit$predAll <- TRUE
spred <- predict(fit,xtest)$s
# Plot
plot(ytest,ypred,xlab="true value",ylab="predicted value",
    pch=20,xlim=c(0.3,1),ylim=c(min(ypred)-0.1,max(ypred)+0.1))
segments(ytest, ypred-spred,ytest, ypred+spred)
epsilon = 0.02
segments(ytest-epsilon,ypred-spred,ytest+epsilon,ypred-spred)
segments(ytest-epsilon,ypred+spred,ytest+epsilon,ypred+spred)
abline(0,1,lty=2)
# Use a different/custom optimizer (here: SANN) for maximum likelihood estimation: 
# (Note: Bound constraints are recommended, to avoid Inf values.
# This is really just a demonstration. SANN does not respect bound constraints.)
optimizer1 <- function(x,fun,lower=NULL,upper=NULL,control=NULL,...){
  res <- optim(x,fun,method="SANN",control=list(maxit=100),...)
  list(xbest=res$par,ybest=res$value,count=res$counts)
}
fit <- modelKriging(x,y,distancePermutationHamming,
                   control=list(algTheta=optimizer1,useLambda=FALSE))
#One-dimensional optimizer (Brent). Note, that Brent will not work when 
#several parameters have to be set, e.g., when using nugget effect (lambda).
#However, Brent may be quite efficient otherwise.
optimizer2 <- function(x,fun,lower,upper,control=NULL,...){
 res <- optim(x,fun,method="Brent",lower=lower,upper=upper,...)
 list(xbest=res$par,ybest=res$value,count=res$counts)
}
fit <- modelKriging(x,y,distancePermutationHamming,
                    control=list(algTheta=optimizer2,useLambda=FALSE))