monmlp.fit {monmlp}

Fit one or more MLP or MONMLP models

Description

Fit an individual model or ensemble of MLP or MONMLP regression models using optimx optimization routines to minimize a least squares cost function. Optional stopped training and bootstrap aggregation (bagging) can be used to help avoid overfitting.

If invoked, the monotone argument enforces increasing behaviour between specified columns of x and model outputs. In this case, the exp function is applied to the relevant weights following initialization and during optimization; manual adjustment of init.weights may be needed.

Note: x and y are automatically standardized prior to fitting and predictions are automatically rescaled by monmlp.predict. This behaviour can be suppressed for y by the scale.y argument.

Usage

monmlp.fit(x, y, hidden1, hidden2 = 0, iter.max = 5000,
           n.trials = 1, n.ensemble = 1, bag = FALSE,
           cases.specified = NULL, iter.stopped = NULL,
           scale.y = TRUE, Th = tansig, To = linear,
           Th.prime = tansig.prime, To.prime = linear.prime,
           monotone = NULL, init.weights = NULL,
           max.exceptions = 10, silent = FALSE, method = "BFGS",
           control = list(trace = 0))

Arguments

x

covariate matrix with number of rows equal to the number of samples and number of columns equal to the number of covariates.

y

response matrix with number of rows equal to the number of samples and number of columns equal to the number of response variables.

hidden1

number of hidden nodes in the first hidden layer.

hidden2

number of hidden nodes in the second hidden layer.

iter.max

maximum number of iterations of the optimization algorithm.

n.trials

number of repeated trials used to avoid local minima.

n.ensemble

number of ensemble members to fit.

bag

logical variable indicating whether or not bootstrap aggregation (bagging) should be used.

cases.specified

if bag = TRUE, a list that specifies the bootstrapped cases to be used in each ensemble member.

iter.stopped

if bag = TRUE, specifies the number of stopped training iterations between calculation of the cost function on the out-of-bootstrap cases.

scale.y

logical determining if columns of the response matrix should be scaled to zero mean and unit variance prior to fitting. Set this to FALSE if using an output layer transfer function that limits the range of predictions.

Th

hidden layer transfer function.

To

output layer transfer function.

Th.prime

derivative of the hidden layer transfer function.

To.prime

derivative of the output layer transfer function.

monotone

column indices of covariates for which the monotonicity constraint should hold.

init.weights

either a vector giving the minimum and maximum allowable values of the random weights, an initial weight vector, or NULL to calculate based on fan-in.

max.exceptions

maximum number of exceptions of the optimization routine before fitting is terminated with an error.

silent

logical determining if diagnostic messages should be suppressed.

method

optimx optimization method.

control

list of optimx control parameters.

Value

list containing fitted weight matrices with attributes including called values of x, y, Th, To, Th.prime, To.prime, monotone, bag, iter.max, and iter.stopped, along with values of covariate/response column means and standard deviations (x.center, x.scale, y.center, y.scale), out-of-bootstrap cases oob, predicted values y.pred, and, if stopped training is switched on, the iteration iter.best and value of the cost function cost.best that minimized the out-of-bootstrap validation error.

See Also

monmlp.predict, gam.style

Examples

set.seed(123)
x <- as.matrix(seq(-10, 10, length = 100))
y <- logistic(x) + rnorm(100, sd = 0.2)

dev.new()
plot(x, y)
lines(x, logistic(x), lwd = 10, col = "gray")

## MLP w/ 2 hidden nodes
w.mlp <- monmlp.fit(x = x, y = y, hidden1 = 2, iter.max = 500)
lines(x, attr(w.mlp, "y.pred"), col = "red", lwd = 3)

## MLP w/ 2 hidden nodes and stopped training
w.stp <- monmlp.fit(x = x, y = y, hidden1 = 2, bag = TRUE,
                    iter.max = 500, iter.stopped = 10)
lines(x, attr(w.stp, "y.pred"), col = "orange", lwd = 3)

## MONMLP w/ 2 hidden nodes
w.mon <- monmlp.fit(x = x, y = y, hidden1 = 2, monotone = 1,
                    iter.max = 500)
lines(x, attr(w.mon, "y.pred"), col = "blue", lwd = 3)