nnet {BoomSpikeSlab}

Bayesian Feed Forward Neural Networks

Description

Fit a feed forward neural network using MCMC.

Usage

BayesNnet(formula,
          hidden.layers,
          niter,
          data,
          subset,
          prior = NULL,
          expected.model.size = Inf,
          drop.unused.levels = TRUE,
          contrasts = NULL,
          ping = niter / 10,
          seed = NULL) 

HiddenLayer(number.of.nodes, prior = NULL, expected.model.size = Inf)

Arguments

formula

A formula describing the model to be fit. The formula should be additive. The network will figure out any interactions or nonlinearities.

hidden.layers

A list of objects created by HiddenLayer defining the network structure. The input layer is determined by the formula argument. The terminal layer is a linear regression on the outputs of the final hidden layer.

niter

The number of MCMC iterations to run. Be sure to include enough so you can throw away a burn-in set.

data

An optional data frame, list or environment (or object coercible by 'as.data.frame' to a data frame) containing the variables in the model. If not found in 'data', the variables are taken from 'environment(formula)', typically the environment from which BayesNnet is called.

subset

an optional vector specifying a subset of observations to be used in the fitting process.

prior

When passed to BayesNnet this is the prior distribution for the terminal layer, which must be an object of class SpikeSlabPrior, SpikeSlabPriorDirect, or NULL. If NULL then a default prior will be used.

When passed to HiddenLayer this is the prior distribution for the coefficients to that layer. The prior is specified for a single output node, and the same prior is used for all nodes. You can think of each hidden layer output node as a logistic regression model where the predictors are the outputs of the previous layer. This must be an object of class MvnPrior, SpikeSlabGlmPrior, or SpikeSlabGlmPriorDirect.

expected.model.size

When prior is not specified a default spike-and-slab prior will be used. The expected.model.size argument to BayesNnet is passed to SpikeSlabPriorDirect. In HiddenLayer the argument is passed to SpikeSlabGlmPriorDirect.

The parameter is used to set the prior inclusion probabilities for the coefficients. If p coefficients are available then the prior inclusion probabilities are each set to expected.model.size / p. If this ratio exceeds 1 then model selection is turned off and all coefficients are included.

drop.unused.levels

Logical indicating whether unobserved factor levels should be dropped when forming the model matrix.

contrasts

An optional list. See the contrasts.arg argument of model.matrix.default.

ping

The frequency with which to print status update messages to the screen. For example, if ping == 10 then an update will be printed every 10 MCMC iterations.

seed

An integer to use as the random seed for the underlying C++ code. If NULL then the seed will be set using the clock.

number.of.nodes

The number of nodes in this hidden layer. This must be a positive scalar integer.

Details

The model is a feedforward neural network regression. The model is fit using an MCMC algorithm based on data augmentation. Each hidden node is randomly assigned a 0/1 value from its full conditional distribution. Then conditional on the imputed data an MCMC draw is done on each latent logistic regression and on the regression model defining the terminal node.

Value

The returned object is a list with class BayesNnet. It contains the following objects

• residual.sd The standard deviation of the residuals from the model.

• hidden.layer.coefficients A list, with one element per hidden layer, giving the posterior draws of the hidden layer coefficients for that layer. Each list element is a 3-way array with dimensions corresponding to

  1. MCMC iteration

  2. Input node. For the first hidden layer each 'input node' is a predictor variable.

  3. Output node.

  You can think of hidden.layer.coefficients[[i]][, , j] as the posterior distribution of the logistic regression model defining node 'j' in hidden layer 'i'.

• terminal.layer.coefficients A matrix containing the MCMC draws of the model coefficients for the terminal layer.

• Other list elements needed to implement various methods (predict, plot, etc.).

Author(s)

Steven L. Scott

References

??

See Also

plot.BayesNnet, predict.BayesNnet.

Examples

if (require(mlbench)) {
  data(BostonHousing)
  hidden.layers <- list(
    HiddenLayer(10, expected.model.size = Inf))

  ## In real life you'd want more 50 MCMC draws.
  model <- BayesNnet(medv ~ .,
    hidden.layers = hidden.layers,
    niter = 50,
    data = BostonHousing)

  par(mfrow = c(1, 2))
  plot(model)  # plots predicted vs actual.
  plot(model, "residual") # plots 
  par(mfrow = c(1,1))
  plot(model, "structure")
  ## Examine all partial dependence plots.
  plot(model, "partial", pch = ".")

  ## Examine a single partial dependence plot.
  par(mfrow = c(1,1))
  plot(model, "lstat", pch = ".")

  ## Check out the mixing performance.
  PlotManyTs(model$terminal.layer.coefficients)
  PlotMacf(model$terminal.layer.coefficients)

  ## Get the posterior distribution of the function values for the
  ## training data.
  pred <- predict(model)

  ## Get predictions for data at new points (though in this example I'm
  ## reusing old points.
  pred2 <- predict(model, newdata = BostonHousing[1:12, ])

} else {
  cat("The Boston housing data from 'mlbench' is needed for this example.")
}