admm.enet {ADMM} | R Documentation |
Elastic Net Regularization
Description
Elastic Net regularization is a combination of \ell_2
stability and
\ell_1
sparsity constraint simulatenously solving the following,
\textrm{min}_x ~ \frac{1}{2}\|Ax-b\|_2^2 + \lambda_1 \|x\|_1 + \lambda_2 \|x\|_2^2
with nonnegative constraints \lambda_1
and \lambda_2
. Note that if both lambda values are 0,
it reduces to least-squares solution.
Usage
admm.enet(
A,
b,
lambda1 = 1,
lambda2 = 1,
rho = 1,
abstol = 1e-04,
reltol = 0.01,
maxiter = 1000
)
Arguments
A |
an |
b |
a length- |
lambda1 |
a regularization parameter for |
lambda2 |
a regularization parameter for |
rho |
an augmented Lagrangian parameter |
abstol |
absolute tolerance stopping criterion |
reltol |
relative tolerance stopping criterion |
maxiter |
maximum number of iterations |
Value
a named list containing
- x
a length-
n
solution vector- history
dataframe recording iteration numerics. See the section for more details.
Iteration History
When you run the algorithm, output returns not only the solution, but also the iteration history recording following fields over iterates,
- objval
object (cost) function value
- r_norm
norm of primal residual
- s_norm
norm of dual residual
- eps_pri
feasibility tolerance for primal feasibility condition
- eps_dual
feasibility tolerance for dual feasibility condition
In accordance with the paper, iteration stops when both r_norm
and s_norm
values
become smaller than eps_pri
and eps_dual
, respectively.
Author(s)
Xiaozhi Zhu
References
Zou H, Hastie T (2005). “Regularization and variable selection via the elastic net.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(2), 301–320. ISSN 1369-7412, 1467-9868, doi: 10.1111/j.1467-9868.2005.00503.x.
See Also
Examples
## generate underdetermined design matrix
m = 50
n = 100
p = 0.1 # percentange of non-zero elements
x0 = matrix(Matrix::rsparsematrix(n,1,p))
A = matrix(rnorm(m*n),nrow=m)
for (i in 1:ncol(A)){
A[,i] = A[,i]/sqrt(sum(A[,i]*A[,i]))
}
b = A%*%x0 + sqrt(0.001)*matrix(rnorm(m))
## run example with both regularization values = 1
output = admm.enet(A, b, lambda1=1, lambda2=1)
niter = length(output$history$s_norm)
history = output$history
## report convergence plot
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(1:niter, history$objval, "b", main="cost function")
plot(1:niter, history$r_norm, "b", main="primal residual")
plot(1:niter, history$s_norm, "b", main="dual residual")
par(opar)