sieve.sgd.solver {Sieve}

Fit sieve-SGD estimators, using progressive validation for hyperparameter tuning.

Description

Fit sieve-SGD estimators, using progressive validation for hyperparameter tuning.

Usage

sieve.sgd.solver(sieve.model, X, Y, cv_weight_rate = 1)

Arguments

sieve.model

a list initiated using sieve.sgd.preprocess. Check the documentation of sieve.sgd.preprocess for more information.

X

a data frame containing prediction features/ independent variables.

Y

training outcome.

cv_weight_rate

this governs the divergence rate of rolling validation statistics. Default is set to be 1 and in general does not need to be changed.

Value

A list. It contains the fitted regression coefficients and progressive validation statistics for each hyperparameter combination.

s.size.sofar

a number. Number of samples has been processed so far.

type

a string. The type of basis funtion.

hyper.para.list

a list of hyperparameters.

index.matrix

a matrix. Identifies the multivariate basis functions used in fitting.

index.row.prod

the index product for each basis function. It is used in calculating basis function - specific learning rates.

inf.list

a list storing the fitted results. It has a length of "number of unique combinations of the hyperparameters". The component of inf.list is itself a list, it has a hyper.para.index domain to specify its corresponding hyperparameters (need to be used together with hyper.para.list). Its rolling.cv domain is the progressive validation statistics for hyperparameter tuning; beta.f is the regression coefficients for the first length(beta.f) basis functions, the rest of the basis have 0 coefficients.

norm_para

a matrix. It records how each dimension of the feature/predictor is rescaled, which is useful when rescaling the testing sample's predictors.

Examples

frho.para <- xdim <- 1 ##predictor dimension
frho <- 'additive' ###truth is a sum of absolute functions 
type <- 'cosine' ###use cosine functions as the basis functions
#generate training data
TrainData <- GenSamples(s.size = 1e3, xdim = xdim, 
                                frho.para = frho.para, 
                                frho = frho, noise.para = 0.1)
#preprocess the model
sieve.model <- sieve.sgd.preprocess(X = TrainData[,2:(xdim+1)], 
                                    type = type,
                                    s = c(1,2),
                                    r0 = c(0.5, 2, 4),
                                    J = c(1, 4, 8))

##train the model
sieve.model <- sieve.sgd.solver(sieve.model = sieve.model, 
                                X = TrainData[,2:(xdim+1)], 
                                Y  = TrainData[,1])

##sieve-SGD can do multiple passes over the data, just like other SGD methods.
##usually a second pass can still improve the prediction accuracy
##watch out overfitting when performing multiple passes!
sieve.model <- sieve.sgd.solver(sieve.model = sieve.model, 
                              X = TrainData[,2:(xdim+1)], 
                              Y  = TrainData[,1])