variational_CAM {SANvi}

Mean Field Variational Bayes estimation of CAM

Description

variational_CAM is used to perform posterior inference under the common atoms model (CAM) of Denti et al. (2023) with Gaussian likelihood. The model uses Dirichlet process mixtures (DPM) at both the observational and distributional levels.

Usage

variational_CAM(y, group, maxL = 30, maxK = 20, 
                m0 = 0, tau0 = .01, lambda0 = 3, gamma0 = 2, 
                conc_hyperpar = c(1,1,1,1), conc_par = NULL, 
                epsilon = 1e-6, seed = NULL, maxSIM = 1e5, 
                warmstart = TRUE, verbose = FALSE)

Arguments

y

Numerical vector of observations (required).

group

Numerical vector of the same length of y, indicating the group membership (required).

maxL, maxK

integers, the upper bounds for the observational and distributional clusters to fit, respectively.

m0, tau0, lambda0, gamma0

Hyperparameters on (\mu, \sigma^2) \sim NIG(m_0, \tau_0, \lambda_0,\gamma_0).

conc_hyperpar, conc_par

Vectors of values used for the concentration parameters of of the stick-breaking representation for the distributional and observational DPs, respectively. The following two arguments can be passed. Specifically,

conc_hyperpar

a vector with 4 positive entries: (s_1^\alpha,s_2^\alpha,s_1^\beta,s_2^\beta). If a random concentration parameters \alpha and \beta are adopted, the specifications are \alpha \sim Gamma(s_1^\alpha,s_2^\alpha) and \beta \sim Gamma(s_1^\beta,s_2^\beta). Default set to unitary vector.

conc_par

a vector with 2 positive entries: (\alpha,\beta). Default is set to NULL. If specified, the previous argument is ignored and the two concentration parameters are assumed fixed and equal to (alpha,beta).

epsilon

the tolerance that drives the convergence criterion adopted as stopping rule

seed

random seed to control the initialization.

maxSIM

the maximum number of CAVI iteration to perform.

warmstart

logical, if TRUE, the observational means of the cluster atoms are initialized with a k-means algorithm.

verbose

logical, if TRUE the iterations are printed.

Details

The common atoms mixture model is used to perform inference in nested settings, where the data are organized into J groups. The data should be continuous observations (Y_1,\dots,Y_J), where each Y_j = (y_{1,j},\dots,y_{n_j,j}) contains the n_j observations from group j, for j=1,\dots,J. The function takes as input the data as a numeric vector y in this concatenated form. Hence y should be a vector of length n_1+\dots+n_J. The group parameter is a numeric vector of the same size as y indicating the group membership for each individual observation. Notice that with this specification the observations in the same group need not be contiguous as long as the correspondence between the variables y and group is maintained.

Model

The data are modeled using a Gaussian likelihood, where both the mean and the variance are observational-cluster-specific, i.e.,

y_{i,j}\mid M_{i,j} = l \sim N(\mu_l,\sigma^2_l)

where M_{i,j} \in \{1,2,\dots\} is the observational cluster indicator of observation i in group j. The prior on the model parameters is a Normal-Inverse-Gamma distribution (\mu_l,\sigma^2_l)\sim NIG (m_0,\tau_0,\lambda_0,\gamma_0), i.e., \mu_l\mid\sigma^2_l \sim N(m_0, \sigma^2_l / \tau_0), 1/\sigma^2_l \sim Gamma(\lambda_0, \gamma_0) (shape, rate).

Clustering

The model performs a clustering of both observations and groups. The clustering of groups (distributional clustering) is provided by the allocation variables S_j \in \{1,2,\dots\}, with

Pr(S_j = k \mid \dots ) = \pi_k \qquad \text{for } \: k = 1,2,\dots

The distribution of the probabilities is \{\pi_k\}_{k=1}^{\infty} \sim GEM(\alpha), where GEM is the Griffiths-Engen-McCloskey distribution of parameter \alpha, which characterizes the stick-breaking construction of the DP (Sethuraman, 1994).

The clustering of observations (observational clustering) is provided by the allocation variables M_{i,j} \in \{1,2,\dots\}, with

Pr(M_{i,j} = l \mid S_j = k, \dots ) = \omega_{l,k} \qquad \text{for } \: k = 1,2,\dots \, ; \: l = 1,2,\dots

The distribution of the probabilities is \{\omega_{l,k}\}_{l=1}^{\infty} \sim GEM(\beta) for all k = 1,2,\dots

Value

variational_CAM returns a list of class SANvb containing four objects:

• model: name of the fitted model.

• params: list containing the data and the parameters used in the simulation. Details below.

• sim: list containing the simulated values (optimized variational parameters). Details below.

• time: total computation time.

Data and parameters: params is a list with the following components:

y, group, Nj, J

Data, group labels, group frequencies, and number of groups.

K, L

Number of fitted distributional and observational clusters.

m0, tau0, lambda0, gamma0

Model hyperparameters.

epsilon, seed

The threshold controlling the convergence criterion and the random seed adopted to replicate the run.

(hyp_alpha1,hyp_alpha2) or alpha

Hyperparameters on \alpha (if \alpha random); or provided value for \alpha (if fixed).

(hyp_beta1,hyp_beta2) or beta

Hyperparameters on \beta (if \beta random); or provided value for \beta (if fixed).

Simulated values: sim is a list with the following components:

theta_l

Matrix of size (L,4). Each row is a posterior variational estimate of the four normal-inverse gamma hyperparameters.

Elbo_val

Vector containing the values of the ELBO.

XI

A list of length J. Each element is a matrix of size (N, L) posterior variational probability of assignment of assignment of the i-th observation in the j-th group to the l-th OC, i.e., \hat{\xi}_{i,j,l} = \hat{\mathbb{Q}}(M_{i,j}=l).

RHO

Matrix of size (J, K). Each row is a posterior variational probability of assignment of the j-th group to the k-th DC, i.e., \hat{\rho}_{j,k} = \hat{\mathbb{Q}}(S_j=k).

a_tilde_k,b_tilde_k

Vector of updated variational parameters of the Beta distributions governing the distributional stick-breaking process.

a_tilde_lk,b_tilde_lk

Matrix of updated variational parameters of the Beta distributions governing the observational stick-breaking process (arranged by column).

conc_hyper

If the concentration parameters are chosen to be random, these object contain a vector with the four updated hyperparameters.

alpha,beta

If the concentration parameters are chosen to be fixed, these objects contain the passed values.

References

Denti, F., Camerlenghi, F., Guindani, M., and Mira, A. (2023). A Common Atoms Model for the Bayesian Nonparametric Analysis of Nested Data. Journal of the American Statistical Association, 118(541), 405-416. DOI: 10.1080/01621459.2021.1933499

Sethuraman, A.J. (1994). A Constructive Definition of Dirichlet Priors, Statistica Sinica, 4, 639–650.

Examples

set.seed(123)
y <- c(rnorm(100),rnorm(100,5))
g <- rep(1:2,rep(100,2))
est <- variational_CAM(y, g, verbose = FALSE,epsilon = 1e-2)