Kronecker covariance {tensorEVD}

Kronecker variance matrix penalization

Description

Ridge penalization of a Kronecker covariance matrix

Usage

Kronecker_cov(K, Sigma = 1, Theta, byrow = FALSE,
              rows = NULL, cols = NULL, drop = TRUE,
              inplace = FALSE)
              

Arguments

K

(numeric) Variance matrix among subjects

Sigma

(numeric) A variance matrix among features. Default Sigma=NULL will consider an identity matrix with the same dimension as Theta

Theta

(numeric) A diagonal-shifting parameter, value to be added to the diagonals of the Kronecker variance matrix. It can be a (symmetric) matrix with the same dimension as Sigma for within (diagonal) and between (off-diagonal) features shifting

byrow

(logical) If FALSE (default) the output Kronecker covariance matrix corresponds to a vectorized random matrix stacked by columns, otherwise, it is assumed to be stacked by rows

rows

(integer) Index which rows of the Kronecker variance are to be returned. Default rows=NULL will return all the rows

cols

(integer) Index which columns of the Kronecker variance are to be returned. Default cols=NULL return all the columns

drop

Either TRUE or FALSE to whether return a uni-dimensional vector when output is a matrix with either 1 row or 1 column as per the rows and cols arguments

inplace

TRUE or FALSE to whether operate directly on matrix K when Sigma and Theta are scalars. This is possible only when rows=NULL and cols=NULL. When TRUE the output will be overwritten on the same address occupied by K. Default inplace=FALSE

Details

Assume that a multi-variate random matrix X with n subjects in rows and p features in columns follows a matrix Gaussian distribution with certain matrix of means M and variance-covariance matrix K of dimension n × n between subjects, and Σ of dimension p × p between features, then its vectorized form vec(X) will also follow a Gaussian distribution with mean vec(M) and variance covariance matrix equal to the Kronecker

Σ⊗K

if the random matrix is vectorized column-wise or

K⊗Σ

if the random matrix is vectorized row-wise.

In the uni-variate case, the problem of near-singularity can be alleviated by penalizing the variance matrix K by adding positive elements θ to its diagonal, i.e., K + θI, where I is an identity matrix. The same can be applied to the multi-variate case where the Kronecker variance matrix is penalized with Θ={θ_ij} of dimensions p × p, where diagonal entries will penalize within feature i and off-diagonals will penalize between features i and j. This is,

Σ⊗K + Θ⊗I

if the random matrix is vectorized column-wise or

K⊗Σ + I⊗Θ

if the random matrix is vectorized row-wise.

Specific rows and columns from this Kronecker can be obtained as per the rows and cols arguments without forming the whole Kronecker product (see help(Kronecker)).

Value

Returns the penalized Kronecker covariance matrix. It can be a sub-matrix of it as per the rows and cols arguments.

Examples

  require(tensorEVD)
  
  # Random matrix witn n subjects in rows and p features in columns
  n = 20
  p = 5
  X = matrix(rnorm(n*p), ncol=p)
  
  # Variance matrix among rows/columns
  K = tcrossprod(X)      # for rows
  Sigma = crossprod(X)   # for columns
  dim(K)      # n x n matrix
  dim(Sigma)  # p x p matrix
  
  # Several examples of penalizing the Kronecker variance
  # ==============================================
  # Example 1. Add unique value 
  # ==============================================
  theta = 10.0
  G = Kronecker_cov(K, Sigma, Theta = theta)

  # it must equal to:
  I0 = diag(n)    # diagonal matrix of dimension n
  Theta0 = matrix(theta, nrow=p, ncol=p)
  G0 = kronecker(Sigma, K) + kronecker(Theta0, I0)
  all.equal(G,G0)
  
  # ==============================================
  # Example 2. Add feature-specific value 
  # ==============================================
  theta = rnorm(p)^2    # One value for each feature
  G = Kronecker_cov(K, Sigma, Theta = theta)

  # it must equal to:
  Theta0 = diag(theta)
  G0 = kronecker(Sigma, K) + kronecker(Theta0, I0)
  all.equal(G,G0)
  
  # ==============================================
  # Example 3. Add specific values within same feature
  #            and between different features 
  # ==============================================
  Theta = crossprod(matrix(rnorm(p*p), ncol=p))
  G = Kronecker_cov(K, Sigma, Theta = Theta)

  # it must equal to:
  G0 = kronecker(Sigma, K) + kronecker(Theta, I0)
  all.equal(G,G0)
  
  # Assume that random matrix X is stacked row-wise
  G = Kronecker_cov(K, Sigma, Theta = Theta, byrow = TRUE)

  # in this case the kronecker is inverted:
  G0 = kronecker(K, Sigma) + kronecker(I0, Theta)
  all.equal(G,G0)
  
  # ==============================================
  # Extra: Selecting specific entries of the output
  # ==============================================
  n = 150
  p = 120
  X = matrix(rnorm(n*p), ncol=p)
  K = tcrossprod(X)      
  Sigma = crossprod(X)   
  Theta = crossprod(matrix(rnorm(p*p), ncol=p))
  
  # We want only some rows and columns
  rows = c(1,3,5)
  cols = c(10,30,50)
  G = Kronecker_cov(K, Sigma, Theta = Theta, rows=rows, cols=cols)

  # this is preferable instead of:
  # I0 = diag(n)
  # G0 = (kronecker(Sigma, K) + kronecker(Theta, I0))[rows,cols]
  # all.equal(G,G0)