cv_SSGL {SSGL}

Cross-Validation for Spike-and-Slab Group Lasso in Group-Regularized Generalized Linear Models (GLMs)

Description

This function implements K-fold cross-validation for group-regularized GLMs with the spike-and-slab group lasso (SSGL) penalty of Bai et al. (2022) and Bai (2023). The identity link function is used for Gaussian regression, the logit link is used for binomial regression, and the log link is used for Poisson, negative binomial, and gamma regression.

Although one can choose lambda0 from cross-validation with this function, it can be very time-consuming to do so if the number of groups G and/or the number of total covariantes p is moderate to large. It is strongly recommended that the user simply run the SSGL function on the training dataset and select the final model according to the lambda0 that minimizes the generalized information criterion (GIC). See description of the SSGL function for more details.

Usage

cv_SSGL(Y, X, groups, 
        family=c("gaussian","binomial","poisson","negativebinomial","gamma"), 
        nb_size=1, gamma_shape=1, group_weights, n_folds=5, n_lambda0=25,
        lambda0, lambda1=1, a=1, b=dim(X)[2], 
        max_iter=100, tol=1e-6, print_fold=TRUE) 

Arguments

Y

n \times 1 vector of responses for training data.

X

n \times p design matrix for training data, where the jth column corresponds to the jth overall feature.

groups

p-dimensional vector of group labels. The jth entry in groups should contain either the group number or the factor level name that the feature in the jth column of X belongs to. groups must be either a vector of integers or factors.

family

exponential dispersion family of the response variables. Allows for "gaussian", "binomial", "poisson", "negativebinomial", and "gamma". Note that for "negativebinomial", the size parameter must be specified in advance, while for "gamma", the shape parameter must be specified in advance.

nb_size

known size parameter \alpha in NB(\alpha,\mu_i) distribution for the responses if the user specifies family="negativebinomial". Default is nb_size=1. Ignored if family is not "negativebinomial".

gamma_shape

known shape parameter \nu in G(\mu_i,\nu) distribution for the responses if the user specifies family="gamma". Default is gamma_shape=1. Ignored if family is not "gamma".

group_weights

group-specific, nonnegative weights for the penalty. Default is to use the square roots of the group sizes.

n_folds

number of folds K to use in K-fold cross-validation. Default is n_folds=5.

n_lambda0

number of spike hyperparameters L. Default is n_lambda0=25.

lambda0

grid of L spike hyperparameters \lambda_0. The user may specify either a scalar or a vector. If the user does not provide this, the program chooses the grid automatically.

lambda1

slab hyperparameter \lambda_1 in the SSGL prior. Default is lambda1=1.

a

shape hyperparameter for the Beta(a,b) prior on the mixing proportion in the SSGL prior. Default is a=1.

b

shape hyperparameter for the Beta(a,b) prior on the mixing proportion in the SSGL prior. Default is b=dim(X)[2].

max_iter

maximum number of iterations in the algorithm. Default is max_iter=100.

tol

convergence threshold for algorithm. Default is tol=1e-6.

print_fold

Boolean variable for whether or not to print the current fold in the algorithm. Default is print_fold=TRUE.

Value

The function returns a list containing the following components:

lambda0

L \times 1 vector of spike hyperparameters lambda0 used to fit the model. lambda0 is displayed in descending order.

cve

L \times 1 vector of mean cross-validation error across all K folds. The kth entry in cve corresponds to the kth spike hyperparameter parameter in lambda0.

cvse

L \times 1 vector of standard errors for cross-validation error across all K folds. The kth entry in cvse corresponds to the kth spike hyperparameter parameter in lambda0.

lambda0_min

The value in lambda0 that minimizes mean cross-validation error cve.

min_index

The index of lambda0_min in lambda0.

References

Bai, R. (2023). "Bayesian group regularization in generalized linear models with a continuous spike-and-slab prior." arXiv pre-print arXiv:2007.07021.

Bai, R., Moran, G. E., Antonelli, J. L., Chen, Y., and Boland, M.R. (2022). "Spike-and-slab group lassos for grouped regression and sparse generalized additive models." Journal of the American Statistical Association, 117:184-197.

Examples

## Generate data
set.seed(12345)
X = matrix(runif(50*6), nrow=50)
n = dim(X)[1]
groups = c(1,1,1,2,2,2)
beta_true = c(-2,1,1.5,0,0,0)

## Generate responses from Gaussian distribution
Y = crossprod(t(X), beta_true) + rnorm(n)

## K-fold cross-validation 
## NOTE: If you do not specify lambda0, the function will automatically choose a suitable grid.
ssgl_mods = cv_SSGL(Y, X, groups, family="gaussian", lambda0=seq(from=16,to=4,by=-4))

## Plot cross-validation curve
plot(ssgl_mods$lambda0, ssgl_mods$cve, type="l", xlab="lambda0", ylab="CVE")
## lambda which minimizes mean CVE
ssgl_mods$lambda0_min
ssgl_mods$min_index


## Example with Poisson regression

## Generate count responses
eta = crossprod(t(X), beta_true)
Y = rpois(n,exp(eta))

## K-fold cross-validation 
## NOTE: If you do not specify lambda0, the program will automatically choose a suitable grid.
ssgl_poisson_mods = cv_SSGL(Y, X, groups, family="poisson", lambda0=seq(from=20,to=2,by=-4))

## Plot cross-validation curve
plot(ssgl_poisson_mods$lambda0, ssgl_poisson_mods$cve, type="l", xlab="lambda0", ylab="CVE")
## lambda which minimizes mean CVE
ssgl_poisson_mods$lambda0_min
ssgl_poisson_mods$min_index