set_optim {MachineShop}

Tuning Parameter Optimization

Description

Set the optimization method and control parameters for tuning of model parameters.

Usage

set_optim_bayes(object, ...)

## S3 method for class 'ModelSpecification'
set_optim_bayes(
  object,
  num_init = 5,
  times = 10,
  each = 1,
  acquisition = c("ucb", "ei", "eips", "poi"),
  kappa = stats::qnorm(conf),
  conf = 0.995,
  epsilon = 0,
  control = list(),
  packages = c("ParBayesianOptimization", "rBayesianOptimization"),
  random = FALSE,
  progress = verbose,
  verbose = 0,
  ...
)

set_optim_bfgs(object, ...)

## S3 method for class 'ModelSpecification'
set_optim_bfgs(
  object,
  times = 10,
  control = list(),
  random = FALSE,
  progress = FALSE,
  verbose = 0,
  ...
)

set_optim_grid(object, ...)

## S3 method for class 'TrainingParams'
set_optim_grid(object, random = FALSE, progress = FALSE, ...)

## S3 method for class 'ModelSpecification'
set_optim_grid(object, ...)

## S3 method for class 'TunedInput'
set_optim_grid(object, ...)

## S3 method for class 'TunedModel'
set_optim_grid(object, ...)

set_optim_pso(object, ...)

## S3 method for class 'ModelSpecification'
set_optim_pso(
  object,
  times = 10,
  each = NULL,
  control = list(),
  random = FALSE,
  progress = FALSE,
  verbose = 0,
  ...
)

set_optim_sann(object, ...)

## S3 method for class 'ModelSpecification'
set_optim_sann(
  object,
  times = 10,
  control = list(),
  random = FALSE,
  progress = FALSE,
  verbose = 0,
  ...
)

set_optim_method(object, ...)

## S3 method for class 'ModelSpecification'
set_optim_method(
  object,
  fun,
  label = "Optimization Function",
  packages = character(),
  params = list(),
  random = FALSE,
  progress = FALSE,
  verbose = FALSE,
  ...
)

Arguments

object

input or model object.

...

arguments passed to the TrainingParams method of set_optim_grid from its other methods.

num_init

number of grid points to sample for the initialization of Bayesian optimization.

times

maximum number of times to repeat the optimization step. Multiple sets of model parameters are evaluated automatically at each step of the BFGS algorithm to compute a finite-difference approximation to the gradient.

each

number of times to sample and evaluate model parameters at each optimization step. This is the swarm size in particle swarm optimization, which defaults to floor(10 + 2 * sqrt(length(bounds))).

acquisition

character string specifying the acquisition function as "ucb" (upper confidence bound), "ei" (expected improvement), "eips" (expected improvement per second), or "poi" (probability of improvement).

kappa, conf

upper confidence bound ("ucb") quantile or its probability to balance exploitation against exploration. Argument kappa takes precedence if both are given and multiplies the predictive standard deviation added to the predictive mean in the acquisition function. Larger values encourage exploration of the model parameter space.

epsilon

improvement methods ("ei", "eips", and "poi") parameter to balance exploitation against exploration. Values should be between -0.1 and 0.1 with larger ones encouraging exploration.

control

list of control parameters passed to bayesOpt by set_optim_bayes with package "ParBayesianOptimization", to BayesianOptimization by set_optim_bayes with package "rBayesianOptimization", to optim by set_optim_bfgs and set_optim_sann, and to psoptim by set_optim_pso.

packages

R package or packages to use for the optimization method, or an empty vector if none are needed. The first package in set_optim_bayes is used unless otherwise specified by the user.

random

number of points to sample for a random grid search, or FALSE for an exhaustive grid search. Used when a grid search is specified or as the fallback method for non-numeric model parameters present during other optimization methods.

progress

logical indicating whether to display iterative progress during optimization.

verbose

numeric or logical value specifying the level of progress detail to print, with 0 (FALSE) indicating none and 1 (TRUE) or higher indicating increasing amounts of detail.

fun

user-defined optimization function to which the arguments below are passed in order. An ellipsis can be included in the function definition when using only a subset of the arguments and ignoring others. A tibble returned by the function with the same number of rows as model evaluations will be included in a TrainingStep summary of optimization results; other types of return values will be ignored.

optim

function that takes a numeric vector or list of named model parameters as the first argument, optionally accepts the maximum number of iterations as argument max_iter, and returns a scalar measure of performance to be maximized. Parameter names are available from the grid and bounds arguments described below. If the function cannot be evaluated at a given set of parameter values, then -Inf is returned.

grid

data frame containing a tuning grid of all model parameters.

bounds

named list of lower and upper bounds for each finite numeric model parameter in grid. The types (integer or double) of the original parameter values are preserved in the bounds.

params

list of optimization parameters as supplied to set_optim_method.

monitor

list of the progress and verbose values.

label

character descriptor for the optimization method.

params

list of user-specified model parameters to be passed to fun.

Details

The optimization functions implement the following methods.

set_optim_bayes

Bayesian optimization with a Gaussian process model (Snoek et al. 2012).

set_optim_bfgs

limited-memory modification of quasi-Newton BFGS optimization (Byrd et al. 1995).

set_optim_grid

exhaustive or random grid search.

set_optim_pso

particle swarm optimization (Bratton and Kennedy 2007, Zambrano-Bigiarini et al. 2013).

set_optim_sann

simulated annealing (Belisle 1992). This method depends critically on the control parameter settings. It is not a general-purpose method but can be very useful in getting to good parameter values on a very rough optimization surface.

set_optim_method

user-defined optimization function.

The package-defined optimization functions evaluate and return values of the tuning parameters that are of same type (e.g. integer, double, character) as given in the object grid. Sequential optimization of numeric tuning parameters is performed over a hypercube defined by their minimum and maximum grid values. Non-numeric parameters are optimized with grid searches.

Value

Argument object updated with the specified optimization method and control parameters.

References

Belisle, C. J. P. (1992). Convergence theorems for a class of simulated annealing algorithms on Rd. Journal of Applied Probability, 29, 885–895.

Bratton, D. & Kennedy, J. (2007), Defining a standard for particle swarm optimization. In IEEE Swarm Intelligence Symposium, 2007 (pp. 120-127).

Byrd, R. H., Lu, P., Nocedal, J., & Zhu, C. (1995). A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing, 16, 1190–1208.

Snoek, J., Larochelle, H., & Adams, R.P. (2012). Practical Bayesian Optimization of Machine Learning Algorithms. arXiv:1206.2944 [stat.ML].

Zambrano-Bigiarini, M., Clerc, M., & Rojas, R. (2013). Standard particle swarm optimisation 2011 at CEC-2013: A baseline for future PSO improvements. In IEEE Congress on Evolutionary Computation, 2013 (pp. 2337-2344).

See Also

BayesianOptimization, bayesOpt, optim, psoptim, set_monitor, set_predict, set_strata

Examples

ModelSpecification(
  sale_amount ~ ., data = ICHomes,
  model = TunedModel(GBMModel)
) %>% set_optim_bayes