sparseSVM {sparseSVM} | R Documentation |
Fit sparse linear SVM with lasso or elasti-net regularization
Description
Fit solution paths for sparse linear SVM regularized by lasso or elastic-net over a grid of values for the regularization parameter lambda.
Usage
sparseSVM(X, y, alpha = 1, gamma = 0.1, nlambda=100,
lambda.min = ifelse(nrow(X)>ncol(X), 0.01, 0.05),
lambda, preprocess = c("standardize", "rescale", "none"),
screen = c("ASR", "SR", "none"), max.iter = 1000, eps = 1e-5,
dfmax = ncol(X)+1, penalty.factor=rep(1, ncol(X)), message = FALSE)
Arguments
X |
Input matrix. |
y |
Output vector. Currently the function only supports binary output and converts the output into +1/-1 coding internally. |
alpha |
The elastic-net mixing parameter that controls the relative contribution
from the lasso and the ridge penalty. It must be a number between 0 and 1. |
gamma |
The tuning parameter for huberization smoothing of hinge loss. Default is 0.1. |
nlambda |
The number of lambda values. Default is 100. |
lambda.min |
The smallest value for lambda, as a fraction of lambda.max, the data derived entry value. Default is 0.01 if the number of observations is larger than the number of variables and 0.05 otherwise. |
lambda |
A user-specified sequence of lambda values. Typical usage is to leave
blank and have the program automatically compute a |
preprocess |
Preprocessing technique to be applied to the input. Either
"standardize" (default), "rescale" or "none" (see |
screen |
Screening rule to be applied at each |
max.iter |
Maximum number of iterations. Default is 1000. |
eps |
Convergence threshold. The algorithms continue until the maximum change in the
objective after any coefficient update is less than |
dfmax |
Upper bound for the number of nonzero coefficients. The algorithm exits and
returns a partial path if |
penalty.factor |
A numeric vector of length equal to the number of variables. Each
component multiplies |
message |
If set to TRUE, sparseSVM will inform the user of its progress. This argument is kept for debugging. Default is FALSE. |
Details
The sequence of models indexed by the regularization parameter lambda
is fitted
using a semismooth Newton coordinate descent algorithm. The objective function is defined
to be
\frac{1}{n} \sum hingeLoss(y_i (x_i' w + b)) + \lambda\textrm{penalty}(w).
where
hingeLoss(t) = max(0, 1-t)
and the intercept b
is unpenalized.
The program supports different types of preprocessing techniques. They are applied to
each column of the input matrix X
. Let x be a column of X
. For
preprocess = "standardize"
, the formula is
x' = \frac{x-mean(x)}{sd(x)};
for preprocess = "rescale"
,
x' = \frac{x-min(x)}{max(x)-min(x)}.
The models are fit with preprocessed input, then the coefficients are transformed back to the original scale via some algebra.
Value
The function returns an object of S3 class "sparseSVM"
, which is a list containing:
call |
The call that produced this object. |
weights |
The fitted matrix of coefficients. The number of rows is equal to the number
of coefficients, and the number of columns is equal to |
iter |
A vector of length |
saturated |
A logical flag for whether the number of nonzero coefficients has reached |
lambda |
The sequence of regularization parameter values in the path. |
alpha |
Same as above. |
gamma |
Same as above. |
penalty.factor |
Same as above. |
levels |
Levels of the output class labels. |
Author(s)
Congrui Yi and Yaohui Zeng
Maintainer: Congrui Yi <eric.ycr@gmail.com>
See Also
Examples
X = matrix(rnorm(1000*100), 1000, 100)
b = 3
w = 5*rnorm(10)
eps = rnorm(1000)
y = sign(b + drop(X[,1:10] %*% w + eps))
fit = sparseSVM(X, y)
coef(fit, 0.05)
predict(fit, X[1:5,], lambda = c(0.2, 0.1))