Sigma_estimation {BlockCov}

This function computes an estimator of the covariance matrix and the square root of its inverse and permutes its rows and columns if it is necessary to make the block structure appear.

Description

This function computes an estimator of the covariance matrix and the square root of its inverse and permutes its rows and columns if it is necessary to make the block structure appear.

Usage

Sigma_estimation(E, k = NULL, nb_nn0 = NULL, big = FALSE,
  reorder = FALSE, inv_12 = FALSE, method_k = "Cattell",
  times = 10, method_0 = "Elbow", N = 10)

Arguments

E

the observation matrix such that each of its row has a block structure correlation matrix Sigma to estimate up to a permutation of its columns and rows.

k

numerical or NULL, the rank for the low rank approximation. If NULL the rank is computed using the slope_change function applied on the eigenvalues of the low rank part of Sigma. Default to NULL.

nb_nn0

numerical or NULL, corresponds to the number of non null values to keep in the estimation of the covariance matrix. If NULL the number of non null values is computed using the slope_change function to the Frobenius norm of the difference between the empirical correlation matrix and its estimation with nb_nn0 non null values. Default to NULL.

big

logical, default to FALSE. If the dataset is too big the empirical correlation is calculated by crossprod(E) * 1 / n to fasten the computation

reorder

logical, default to FALSE. Whether or not the columns of E are permuted. If TRUE a hierarchical clustering is first performed and the columns are permuted according to it.

inv_12

logical, default to FALSE Whether or not computing the square root of the inverse of the covariance matrix.

method_k

character if "Cattell" (the default) then the Cattell criterion (Cattell 1966) is performed on the singular values of the covariance matrix. to estimate the number of rank use in the low rank approximation, while "PA" use the parrallel analysis (Horn 1965) wich can be more accurate if the number of rows of E is not to small but which is much slower.

times

numeric the number of resampling done for the "PA" method, ignored if metod_k is different from "PA".

method_0

character if "Elbow" (the default) then the Elbow criterion (see Perrot-Dockès et al. (2018) for details) is performed to estimate the number of rank use in the low rank approximation, while "BL" use the approach proposed in Bickel and Levina (2008) based on cross-validation wich can be more accurate if the number of rows of E is not to small but which is much slower.

N

numeric the number of fold used for the "BL" method. Ignored if method_0 is different from "BL"

Value

A list with the elements

Sigma_est

estimator of the covariance matrix

k

rank of the low rank part of the covariance matrix

nb_nn0

number of non null values of the upper triangular part of the covariance matrix

S_inv_12

square root of the inverse of the estimated covariance matrix

order

permutation to apply to the rows and the columns of the covariance to make the block structure appear

References

Bickel PJ, Levina E (2008). “Covariance regularization by thresholding.” Ann. Statist., 36(6), 2577–2604. doi: 10.1214/08-AOS600, https://doi.org/10.1214/08-AOS600.

Cattell RB (1966). “The scree test for the number of factors.” Multivariate behavioral research, 1(2), 245-276.

Horn JL (1965). “A rationale and test for the number of factors in factor analysis.” Psychometrika, 30(2), 179–185. ISSN 1860-0980, doi: 10.1007/BF02289447, https://doi.org/10.1007/BF02289447.

Perrot-Dockès M, Lévy-Leduc C, Rajjou L (2018). “Estimation of large block structured covariance matrices: Application to "multi-omic" approaches to study seed quality.” arXiv:1806.10093.

Examples

n <- 30
q <- 100
Sigma <- Simu_Sigma(q = q, diag = FALSE, equal = TRUE)
Matrix::image(Sigma)
E <- matrix(rnorm(n * q), ncol = q) %*% chol(as.matrix(Sigma))
res <- Sigma_estimation(E, inv_12 = TRUE)
Matrix::image(res$Sigma_est)
Matrix::image(res$S_inv_12)