ensemble {CVEK} | R Documentation |
Estimating Ensemble Kernel Matrices
Description
Give the ensemble projection matrix and weights of the kernels in the library.
Usage
ensemble(strategy, beta_exp, error_mat, A_hat)
Arguments
strategy |
(character) A character string indicating which ensemble strategy is to be used. |
beta_exp |
(numeric/character) A numeric value specifying the parameter
when strategy = "exp" |
error_mat |
(matrix, n*K) A n\*K matrix indicating errors. |
A_hat |
(list of length K) A list of projection matrices to kernel space for each kernel in the kernel library. |
Details
There are three ensemble strategies available here:
Empirical Risk Minimization (Stacking)
After obtaining the estimated errors \{\hat{\epsilon}_d\}_{d=1}^D
, we
estimate the ensemble weights u=\{u_d\}_{d=1}^D
such that it minimizes
the overall error
\hat{u}={argmin}_{u \in \Delta}\parallel
\sum_{d=1}^Du_d\hat{\epsilon}_d\parallel^2 \quad where\; \Delta=\{u | u \geq
0, \parallel u \parallel_1=1\}
Then produce the final ensemble prediction:
\hat{h}=\sum_{d=1}^D \hat{u}_d h_d=\sum_{d=1}^D \hat{u}_d
A_{d,\hat{\lambda}_d}y=\hat{A}y
where \hat{A}=\sum_{d=1}^D \hat{u}_d
A_{d,\hat{\lambda}_d}
is the ensemble matrix.
Simple Averaging
Motivated by existing literature in omnibus kernel, we propose another way
to obtain the ensemble matrix by simply choosing unsupervised weights
u_d=1/D
for d=1,2,...D
.
Exponential Weighting
Additionally, another scholar gives a new strategy to calculate weights
based on the estimated errors \{\hat{\epsilon}_d\}_{d=1}^D
.
u_d(\beta)=\frac{exp(-\parallel \hat{\epsilon}_d
\parallel_2^2/\beta)}{\sum_{d=1}^Dexp(-\parallel \hat{\epsilon}_d
\parallel_2^2/\beta)}
Value
A_est |
(matrix, n*n) The ensemble projection matrix. |
u_hat |
(vector of length K) A vector of weights of the kernels in the library. |
Author(s)
Wenying Deng
References
Jeremiah Zhe Liu and Brent Coull. Robust Hypothesis Test for Nonlinear Effect with Gaussian Processes. October 2017.
Xiang Zhan, Anna Plantinga, Ni Zhao, and Michael C. Wu. A fast small-sample kernel independence test for microbiome community-level association analysis. December 2017.
Arnak S. Dalalyan and Alexandre B. Tsybakov. Aggregation by Exponential Weighting and Sharp Oracle Inequalities. In Learning Theory, Lecture Notes in Computer Science, pages 97– 111. Springer, Berlin, Heidelberg, June 2007.
See Also
mode: tuning