Selection_HMTL {HMTL}

Model Selection for Multi-task Feature Learning based on Bayesian Information Criterion (BIC)

Description

Selection_HMTL can be used to search the optimal candidate model based on Bayesian Information Criterion (BIC).

Usage

Selection_HMTL(
  y,
  x,
  lambda,
  Kn,
  p,
  n,
  beta = 0.1,
  tau = 1.45,
  Cont_Model = "adaptive Huber",
  type = "Heterogeneity",
  import_w = 1,
  tol = 0.05,
  max_iter = 100,
  Complete = "True",
  diagnostics = FALSE,
  gamma = 1,
  alpha = 1
)

Arguments

y

List. A list of responses vectors for all tasks. The order of the list put the continuous responses before the binary responses.

x

List. Listing matrices of the predictors for all tasks align with the same order as in y.

lambda

Numeric. The penalty parameter used for block-wise regularization (\ell_{2,1} norm).

Kn

Vector of two elements. First element is the number of tasks with continuous responses, and the second element is the number of tasks with binary responses.

p

Numeric. The number of features.

n

Numeric or vector. If only one numeric value is provided, equal sample size will be assumed for each task. If a vector is provided, then the elements are the sample sizes for all tasks.

beta

(optional). Numeric or matrix. An initial value or matrix of values p by K for the estimation. The default value is 0.1.

tau

Numeric or vector. The robustification parameter used for methods "Huber regression" or "Adaptive Huber". The default value is 1.45.

Cont_Model

Character("regression", "Huber regression", or "adaptive Huber"). The models used for tasks with continuous responses.

type

Character("Heterogeneity", "Continuous" or "Binary"). type = "Heterogeneity" used for MTL_hetero; type = "Continuous" used for MTL_reg; and type = "Binary" used for MTL_class.

import_w

Numeric or vector. The weights assigned to different tasks. An equal weight is set as the default.

tol

(optional). Numeric. The tolerance level of optimation.

max_iter

(optional). Numeric. The maximum number of iteration steps.

Complete

Logic input. If the predictors in each task are all measured, set 'Complete == TRUE'; If some predictors in some but not all task are all measured, set'Complete == FALSE', and the missing values are imputed by column mean. The adjustment weights will be assigned based on the completeness of the predictors.

diagnostics

Logic input. If 'diagnostics == TRUE', the function provides Bayesian information criterion, and the selected model performance is evalued by the MSE and MAE for tasks with continuous response and the AUC and deviance for tasks with binary responses.

gamma

(optional). Numeric. Step size for each inner iteration. The default is equal to 1.

alpha

(optional). Numeric. A tuning parameter for BIC penalty. The default is equal to 1.

Details

The Bayesian information criterion is given by

BIC(s) = 2\mathcal{L}_s(\hat{\theta}) + d_s^{*} \gamma_n,

where \hat{\theta} is the estimated coefficients and s is denoted the selected support set of \hat{\theta}. In addition, \mathcal{L}_s(\hat{\theta}) denoted the estimated composite quasi-likelihood or adaptive Huber loss evaluated as the \hat{\theta}. The degree of freedom d_s^{*} can measure the model complexity, which is estimated by tr(H_s^{-1}(\hat{\theta}) J_s(\hat{\theta}) ). The sensitivity matrix and specificity matrix can be given by H(\theta) = E(\nabla^2 \mathcal{L}( {\theta})) and J(\theta) = -Cov(\nabla \mathcal{L}( {\theta})). The penalty term \gamma_n can be defined by users.

Value

A table of Bayesian Information Criterion (BIC)

lambda

A list of penalty parameters.

Compo_likelihood

Sum of empirical loss functions estimated based on the selected parameters .

Degree of freedom

Penalty component based on the selected parameters.

Info criterion

Bayesian Information Criterion (BIC): robust BIC or pseudo BIC.

References

Y. Zhong, W. Xu, and X. Gao (2023) Robust Multi-task Feature Learning. Submitted

Gao, X and Carroll, R. J. (2017) Data integration with high dimensionality. Biometrika, 104, 2, pp. 251-272

Examples

lambda <- c(1.2, 2.0, 2.3, 2.8)
cv.mtl <- Selection_HMTL(mockdata2,mockdata1, lambda =lambda, Kn = c(2,2), p=500,
               n = c(500,250,250,250),gamma = 2, Complete = FALSE, diagnostics = TRUE, alpha = 2)
plot_HMTL(cv.mtl)
cv.mtl$Selection_Results