VariableSelection {BranchGLM} | R Documentation |
Variable Selection for GLMs
Description
Performs forward selection, backward elimination, and efficient best subset variable selection with information criterion for generalized linear models (GLMs). Best subset selection is performed with branch and bound algorithms to greatly speed up the process.
Usage
VariableSelection(object, ...)
## S3 method for class 'formula'
VariableSelection(
object,
data,
family,
link,
offset = NULL,
method = "Fisher",
type = "switch branch and bound",
metric = "AIC",
bestmodels = NULL,
cutoff = NULL,
keep = NULL,
keepintercept = TRUE,
maxsize = NULL,
grads = 10,
parallel = FALSE,
nthreads = 8,
tol = 1e-06,
maxit = NULL,
contrasts = NULL,
showprogress = TRUE,
...
)
## S3 method for class 'BranchGLM'
VariableSelection(
object,
type = "switch branch and bound",
metric = "AIC",
bestmodels = NULL,
cutoff = NULL,
keep = NULL,
keepintercept = TRUE,
maxsize = NULL,
parallel = FALSE,
nthreads = 8,
showprogress = TRUE,
...
)
Arguments
object |
a formula or a |
... |
further arguments. |
data |
a data.frame, list or environment (or object coercible by as.data.frame to a data.frame), containing the variables in formula. Neither a matrix nor an array will be accepted. |
family |
the distribution used to model the data, one of "gaussian", "gamma", "binomial", or "poisson". |
link |
the link used to link the mean structure to the linear predictors. One of "identity", "logit", "probit", "cloglog", "sqrt", "inverse", or "log". |
offset |
the offset vector, by default the zero vector is used. |
method |
one of "Fisher", "BFGS", or "LBFGS". Fisher's scoring is recommended for forward selection and branch and bound methods since they will typically fit many models with a small number of covariates. |
type |
one of "forward", "backward", "branch and bound", "backward branch and bound", or "switch branch and bound" to indicate the type of variable selection to perform. The default value is "switch branch and bound". The branch and bound algorithms are guaranteed to find the best models according to the metric while "forward" and "backward" are heuristic approaches that may not find the optimal model. |
metric |
the metric used to choose the best models, the default is "AIC", but "BIC" and "HQIC" are also available. AIC is the Akaike information criterion, BIC is the Bayesian information criterion, and HQIC is the Hannan-Quinn information criterion. |
bestmodels |
a positive integer to indicate the number of the best models to find according to the chosen metric or NULL. If this is NULL, then cutoff is used instead. This is only used for the branch and bound methods. |
cutoff |
a non-negative number which indicates that the function should return all models that have a metric value within cutoff of the best metric value or NULL. Only one of this or bestmodels should be specified and when both are NULL a cutoff of 0 is used. This is only used for the branch and bound methods. |
keep |
a character vector of names to denote variables that must be in the models. |
keepintercept |
a logical value to indicate whether to keep the intercept in all models, only used if an intercept is included in the formula. |
maxsize |
a positive integer to denote the maximum number of variables to
consider in a single model, the default is the total number of variables.
This number adds onto any variables specified in keep. This argument only works
for |
grads |
a positive integer to denote the number of gradients used to
approximate the inverse information with, only for |
parallel |
a logical value to indicate if parallelization should be used. |
nthreads |
a positive integer to denote the number of threads used with OpenMP,
only used if |
tol |
a positive number to denote the tolerance used to determine model convergence. |
maxit |
a positive integer to denote the maximum number of iterations performed. The default for Fisher's scoring is 50 and for the other methods the default is 200. |
contrasts |
see |
showprogress |
a logical value to indicate whether to show progress updates for branch and bound methods. |
Details
The supplied formula or the formula from the fitted model is treated as the upper model. The variables specified in keep along with an intercept (if included in formula and keepintercept = TRUE) is the lower model. Factor variables are either kept in their entirety or entirely removed and interaction terms are properly handled. All observations that have any missing values in the upper model are removed.
Algorithms
The branch and bound method makes use of an efficient branch and bound algorithm to find the optimal models. This will find the best models according to the metric and can be much faster than an exhaustive search and can be made even faster with parallel computation. The backward branch and bound method is very similar to the branch and bound method, except it tends to be faster when the best models contain most of the variables. The switch branch and bound method is a combination of the two methods and is typically the fastest of the 3 branch and bound methods.
Fisher's scoring is recommended for branch and bound selection and forward selection. L-BFGS may be faster for backward elimination, especially when there are many variables.
Value
A BranchGLMVS
object which is a list with the following components
initmodel |
the |
numchecked |
number of models fit |
names |
character vector of the names of the predictor variables |
order |
the order the variables were added to the model or removed from the model, this is not included for branch and bound selection |
type |
type of variable selection employed |
metric |
metric used to select best models |
bestmodels |
numeric matrix used to describe the best models |
bestmetrics |
numeric vector with the best metrics found in the search |
beta |
numeric matrix of beta coefficients for the best models |
cutoff |
the cutoff that was used, this is set to -1 if bestmodels was used instead |
keep |
vector of which variables were kept through the selection process |
See Also
plot.BranchGLMVS, coef.BranchGLMVS, predict.BranchGLMVS, summary.BranchGLMVS
Examples
Data <- iris
Fit <- BranchGLM(Sepal.Length ~ ., data = Data, family = "gaussian",
link = "identity")
# Doing branch and bound selection
VS <- VariableSelection(Fit, type = "branch and bound", metric = "BIC",
bestmodels = 10, showprogress = FALSE)
VS
## Plotting the BIC of the best models
plot(VS, type = "b")
## Getting the coefficients of the best model according to BIC
FinalModel <- coef(VS, which = 1)
FinalModel
# Now doing it in parallel (although it isn't necessary for this dataset)
parVS <- VariableSelection(Fit, type = "branch and bound", parallel = TRUE,
metric = "BIC", bestmodels = 10, showprogress = FALSE)
## Getting the coefficients of the best model according to BIC
FinalModel <- coef(parVS, which = 1)
FinalModel
# Using a formula
formVS <- VariableSelection(Sepal.Length ~ ., data = Data, family = "gaussian",
link = "identity", metric = "BIC", type = "branch and bound", bestmodels = 10,
showprogress = FALSE)
## Getting the coefficients of the best model according to BIC
FinalModel <- coef(formVS, which = 1)
FinalModel
# Using the keep argument
keepVS <- VariableSelection(Fit, type = "branch and bound",
keep = c("Species", "Petal.Width"), metric = "BIC", bestmodels = 4,
showprogress = FALSE)
keepVS
## Getting the coefficients from the fourth best model according to BIC when
## keeping Petal.Width and Species in every model
FinalModel <- coef(keepVS, which = 4)
FinalModel
# Treating categorical variable beta parameters separately
## This function automatically groups together parameters from a categorical variable
## to avoid this, you need to create the indicator variables yourself
x <- model.matrix(Sepal.Length ~ ., data = iris)
Sepal.Length <- iris$Sepal.Length
Data <- cbind.data.frame(Sepal.Length, x[, -1])
VSCat <- VariableSelection(Sepal.Length ~ ., data = Data, family = "gaussian",
link = "identity", metric = "BIC", bestmodels = 10, showprogress = FALSE)
VSCat
## Plotting results
plot(VSCat, cex.names = 0.75)