VariableSelection {BranchGLM}R Documentation

Variable Selection for GLMs

Description

Performs forward selection, backward elimination, and efficient best subset variable selection with information criterion for generalized linear models (GLMs). Best subset selection is performed with branch and bound algorithms to greatly speed up the process.

Usage

VariableSelection(object, ...)

## S3 method for class 'formula'
VariableSelection(
  object,
  data,
  family,
  link,
  offset = NULL,
  method = "Fisher",
  type = "switch branch and bound",
  metric = "AIC",
  bestmodels = NULL,
  cutoff = NULL,
  keep = NULL,
  keepintercept = TRUE,
  maxsize = NULL,
  grads = 10,
  parallel = FALSE,
  nthreads = 8,
  tol = 1e-06,
  maxit = NULL,
  contrasts = NULL,
  showprogress = TRUE,
  ...
)

## S3 method for class 'BranchGLM'
VariableSelection(
  object,
  type = "switch branch and bound",
  metric = "AIC",
  bestmodels = NULL,
  cutoff = NULL,
  keep = NULL,
  keepintercept = TRUE,
  maxsize = NULL,
  parallel = FALSE,
  nthreads = 8,
  showprogress = TRUE,
  ...
)

Arguments

object

a formula or a BranchGLM object.

...

further arguments.

data

a data.frame, list or environment (or object coercible by as.data.frame to a data.frame), containing the variables in formula. Neither a matrix nor an array will be accepted.

family

the distribution used to model the data, one of "gaussian", "gamma", "binomial", or "poisson".

link

the link used to link the mean structure to the linear predictors. One of "identity", "logit", "probit", "cloglog", "sqrt", "inverse", or "log".

offset

the offset vector, by default the zero vector is used.

method

one of "Fisher", "BFGS", or "LBFGS". Fisher's scoring is recommended for forward selection and branch and bound methods since they will typically fit many models with a small number of covariates.

type

one of "forward", "backward", "branch and bound", "backward branch and bound", or "switch branch and bound" to indicate the type of variable selection to perform. The default value is "switch branch and bound". The branch and bound algorithms are guaranteed to find the best models according to the metric while "forward" and "backward" are heuristic approaches that may not find the optimal model.

metric

the metric used to choose the best models, the default is "AIC", but "BIC" and "HQIC" are also available. AIC is the Akaike information criterion, BIC is the Bayesian information criterion, and HQIC is the Hannan-Quinn information criterion.

bestmodels

a positive integer to indicate the number of the best models to find according to the chosen metric or NULL. If this is NULL, then cutoff is used instead. This is only used for the branch and bound methods.

cutoff

a non-negative number which indicates that the function should return all models that have a metric value within cutoff of the best metric value or NULL. Only one of this or bestmodels should be specified and when both are NULL a cutoff of 0 is used. This is only used for the branch and bound methods.

keep

a character vector of names to denote variables that must be in the models.

keepintercept

a logical value to indicate whether to keep the intercept in all models, only used if an intercept is included in the formula.

maxsize

a positive integer to denote the maximum number of variables to consider in a single model, the default is the total number of variables. This number adds onto any variables specified in keep. This argument only works for type = "forward" and type = "branch and bound".

grads

a positive integer to denote the number of gradients used to approximate the inverse information with, only for method = "LBFGS"..

parallel

a logical value to indicate if parallelization should be used.

nthreads

a positive integer to denote the number of threads used with OpenMP, only used if parallel = TRUE.

tol

a positive number to denote the tolerance used to determine model convergence.

maxit

a positive integer to denote the maximum number of iterations performed. The default for Fisher's scoring is 50 and for the other methods the default is 200.

contrasts

see contrasts.arg of model.matrix.default.

showprogress

a logical value to indicate whether to show progress updates for branch and bound methods.

Details

The supplied formula or the formula from the fitted model is treated as the upper model. The variables specified in keep along with an intercept (if included in formula and keepintercept = TRUE) is the lower model. Factor variables are either kept in their entirety or entirely removed and interaction terms are properly handled. All observations that have any missing values in the upper model are removed.

Algorithms

The branch and bound method makes use of an efficient branch and bound algorithm to find the optimal models. This will find the best models according to the metric and can be much faster than an exhaustive search and can be made even faster with parallel computation. The backward branch and bound method is very similar to the branch and bound method, except it tends to be faster when the best models contain most of the variables. The switch branch and bound method is a combination of the two methods and is typically the fastest of the 3 branch and bound methods.

Fisher's scoring is recommended for branch and bound selection and forward selection. L-BFGS may be faster for backward elimination, especially when there are many variables.

Value

A BranchGLMVS object which is a list with the following components

initmodel

the BranchGLM object corresponding to the upper model

numchecked

number of models fit

names

character vector of the names of the predictor variables

order

the order the variables were added to the model or removed from the model, this is not included for branch and bound selection

type

type of variable selection employed

metric

metric used to select best models

bestmodels

numeric matrix used to describe the best models

bestmetrics

numeric vector with the best metrics found in the search

beta

numeric matrix of beta coefficients for the best models

cutoff

the cutoff that was used, this is set to -1 if bestmodels was used instead

keep

vector of which variables were kept through the selection process

See Also

plot.BranchGLMVS, coef.BranchGLMVS, predict.BranchGLMVS, summary.BranchGLMVS

Examples

Data <- iris
Fit <- BranchGLM(Sepal.Length ~ ., data = Data, family = "gaussian", 
link = "identity")

# Doing branch and bound selection 
VS <- VariableSelection(Fit, type = "branch and bound", metric = "BIC", 
bestmodels = 10, showprogress = FALSE)
VS

## Plotting the BIC of the best models
plot(VS, type = "b")

## Getting the coefficients of the best model according to BIC
FinalModel <- coef(VS, which = 1)
FinalModel

# Now doing it in parallel (although it isn't necessary for this dataset)
parVS <- VariableSelection(Fit, type = "branch and bound", parallel = TRUE, 
metric = "BIC", bestmodels = 10, showprogress = FALSE)

## Getting the coefficients of the best model according to BIC
FinalModel <- coef(parVS, which = 1)
FinalModel

# Using a formula
formVS <- VariableSelection(Sepal.Length ~ ., data = Data, family = "gaussian", 
link = "identity", metric = "BIC", type = "branch and bound", bestmodels = 10, 
showprogress = FALSE)

## Getting the coefficients of the best model according to BIC
FinalModel <- coef(formVS, which = 1)
FinalModel

# Using the keep argument
keepVS <- VariableSelection(Fit, type = "branch and bound", 
keep = c("Species", "Petal.Width"), metric = "BIC", bestmodels = 4, 
showprogress = FALSE)
keepVS

## Getting the coefficients from the fourth best model according to BIC when 
## keeping Petal.Width and Species in every model
FinalModel <- coef(keepVS, which = 4)
FinalModel

# Treating categorical variable beta parameters separately
## This function automatically groups together parameters from a categorical variable
## to avoid this, you need to create the indicator variables yourself
x <- model.matrix(Sepal.Length ~ ., data = iris)
Sepal.Length <- iris$Sepal.Length
Data <- cbind.data.frame(Sepal.Length, x[, -1])
VSCat <- VariableSelection(Sepal.Length ~ ., data = Data, family = "gaussian", 
link = "identity", metric = "BIC", bestmodels = 10, showprogress = FALSE)
VSCat

## Plotting results
plot(VSCat, cex.names = 0.75)


[Package BranchGLM version 2.1.4 Index]