teigen {teigen}

teigen: Function for Model-Based Clustering and Classification with the Multivariate t Distribution

Description

Fits multivariate t-distribution mixture models (with eigen-decomposed covariance structure) to the given data within a clustering paradigm (default) or classification paradigm (by giving either training index or percentage of data taken to be known). Can be run in parallel.

Usage

teigen(x, Gs = 1:9, models = "all", init = "kmeans", scale = TRUE, dfstart = 50, 
	known = NULL, training = NULL, gauss = FALSE, dfupdate = "approx", 
	eps = c(0.001, 0.1), verbose = TRUE, maxit = c(Inf,Inf), 
	convstyle = "aitkens", parallel.cores = FALSE, 
	ememargs = list(25, 5, "UUUU", "hard"))

Arguments

x

A numeric matrix, data frame, or vector (for univariate data) .

Gs

A number or vector indicating the number of groups to fit. Default is 1-9.

models

A character vector giving the models to fit. See details for a comprehensive list of choices.

init

A list of initializing classification of the form that init[[G]] contains the initializing vector for all G considered (see example below). Alternatively, the user can use a character string indicating initialization method. Currently the user can choose from "kmeans" (default), ‘hard’ random - "hard", ‘soft’ random - "soft", ``emem'' (see ememargs below for description), and "uniform" (classification only).

scale

Logical indicating whether or not the function should scale the data. Default is TRUE and is the prescribed method — tEIGEN models are not scale invariant.

dfstart

The initialized value for the degrees of freedom. The default is 50.

training

Optional indexing vector for the observations whose classification is taken to be known.

known

A vector of known classifications that can be numeric or character - optional for clustering, necessary for classification. Must be the same length as the number of rows in the data set. If using in a true classification sense, give samples with unknown classification the value NA within known (see training example below).

gauss

Logical indicating if the algorithm should use the gaussian distribution. If models="mclust" or "gaussian" then gauss=TRUE is forced.

dfupdate

Character string or logical indicating how the degrees of freedom should be estimated. The default is "approx" indicating a closed form approximation be used. Alternatively, "numeric" can be specified which makes use of uniroot. If FALSE, the value from dfstart is used and the degrees of freedom are not updated. If TRUE, "numeric" will be used for back-compatibility.

eps

Vector (of size 2) giving tolerance values for the convergence criterion. First value is the tolerance level for iterated M-steps. Second value is tolerance for the EM algorithm: convergence is based on Aitken's acceleration, see cited papers for more information.

verbose

Logical indicating whether the running output should be displayed. This option is not available in parallel. What is displayed depends on the width of the R window. With a width of 80 or larger: time run, estimated time remaining, percent complete are all displayed.

maxit

Vector (of size 2) giving maximum iteration number for the iterated M-steps and EM algorithm, respectively. A warning is displayed if either of these maximums are met, default for both is Inf (aka, no limit).

convstyle

Character string specifying the method of determining convergence. Default is "aitkens" which uses a criteria based on Aitken's acceleration, but "lop" (lack of progress) may be used instead.

parallel.cores

Logical indicating whether to run teigen in parallel or not. If TRUE, then the function determines the number of cores available and uses all of them. Alternatively, a positive integer may be provided indicating the number of cores the user wishes to use for running in parallel.

ememargs

A list of the controls for the emEM initialization with named elements: numstart - numeric, number of starts (default 25) iter - numeric, number of EM iterations (default 5) model - character string, model name to be used (default “UUUU” from C,U,I nomenclature...see details below) init - character string, initialization method for emEM (default hard, or soft, or kmeans). The emEM initialiazation will run multiple, randomized initialization attempts for a limited number of iterations, and then continue the model-fitting process.

Details

Model specification (via the models argument) follows either the nomenclature discussed in Andrews and McNicholas (2012), or via the nomenclature popularized in other packages. In both cases, the nomenclature refers to the decomposition and constraints on the covariance matrix:

\Sigma_g = \lambda_g D_g A_g D_g'

The nomenclature from Andrews and McNicholas (2012) gives four letters, each letter referring to (in order) \lambda, D, A, and the degrees of freedom. Possible letters are "U" for unconstrained, "C" for constrained (across groups), and "I" for when the parameter is replaced by the appropriately sized identity matrix (where applicable). As an example, the string "UICC" would refer to the model where \Sigma_g = \lambda_g A with degrees of freedom held equal across groups.

The alternative nomenclature describes (in order) the volume (\lambda), shape (A), orientation (D), and degrees of freedom in terms of "V"ariable, "E"qual, or the "I"dentity matrix. The example model discussed in the previous paragraph would then be called by "VEIE".

Possible univariate models are c("univUU", "univUC", "univCU", "univCC") where the first capital letter describes "U"nconstrained or "C"onstrained variance and the second capital letter refers to the degrees of freedom. Once again, "V"ariable or "E"qual can replace U and C, but this time the orders match between the nomenclatures.

As many models as desired can be selected and ran via the vector supplied to models. More commonly, subsets can be called by the following character strings: "all" runs all 28 tEIGEN models (default), "dfunconstrained" runs the 14 unconstrained degrees of freedom models, "dfconstrained" runs the 14 constrained degrees of freedom models, "mclust" runs the 10 MCLUST models using the multivariate Gaussian distribution rather than the multivariate t, "gaussian" is similar but includes four further mixture models than MCLUST, "univariate" runs the univariate models - will automatically be called if one of the previous shortcuts is used on univariate data.

Note that adding "alt" to the beginning of those previously mentioned characters strings will run the same models, but return results with the V-E-I nomenclature.

Also note that for G=1, several models are equivalent (for example, UUUU and CCCC). Thus, for G=1 only one model from each set of equivalent models will be run.

Value

x

Data used for clustering/classification.

index

Indexing vector giving observations taken to be known (only available when classification is performed).

classification

Vector of group classifications as determined by the BIC.

bic

BIC of the best fitted model.

modelname

Name of the best model according to the BIC.

allbic

Matrix of BIC values according to model and G. A value of -Inf is returned when the model did not converge.

bestmodel

Character string giving best model (BIC) details.

G

Value corresponding to the number of components chosen by the BIC.

tab

Classification table for BIC-selected model (only available when known is given). When classification is used the “known” observations are left out of the table.

fuzzy

The fuzzy clustering matrix for the model selected by the BIC.

logl

The log-likelihood corresponding to the model with the best BIC.

iter

The number of iterations until convergence for the model selected by the BIC.

parameters

List containing the fitted parameters: mean - matrix of means where the rows correspond to the component and the columns are the variables; sigma - array of covariance matrices (multivariate) or variances (univariate); lambda - vector of scale parameters, or constants of proportionality; d - eigenvectors, or orientation matrices; a - diagonal matrix proportional to eigenvalues, or shape matrices; df - vector containing the degrees of freedom for each component; weights - matrix of the expected value of the characteristic weights; pig - a vector giving the mixing proportions.

iclresults

List containing all the previous outputs, except x and index, pertaining to the model chosen by the best ICL (all under the same name except allicl and icl are the equivalent of allbic and bic, respectively).

info

List containing a few of the original user inputs, for use by other dedicated functions of the teigen class.

Author(s)

Jeffrey L. Andrews, Jaymeson R. Wickins, Nicholas M. Boers, Paul D. McNicholas

References

Andrews JL and McNicholas PD. “Model-based clustering, classification, and discriminant analysis with the multivariate t-distribution: The tEIGEN family” Statistics and Computing 22(5), 1021–1029.

Andrews JL, McNicholas PD, and Subedi S (2011) “Model-based classification via mixtures of multivariate t-distributions” Computational Statistics and Data Analysis 55, 520–529.

See Also

See package manual tEIGEN

Examples

###Note that only one model is run for each example
###in order to reduce computation time

#Clustering old faithful data with hard random start
tfaith <- teigen(faithful, models="UUUU", Gs=1:3, init="hard")
plot(tfaith, what = "uncertainty")
summary(tfaith)

#Clustering old faithful with hierarchical starting values
initial_list <- list()
clustree <- hclust(dist(faithful))
for(i in 1:3){
	initial_list[[i]] <- cutree(clustree,i)
}
tfaith <- teigen(faithful, models="CUCU", Gs=1:3, init=initial_list)
print(tfaith)

#Classification with the iris data set
#Introducing NAs is not required; this is to illustrate a `true' classification scenario
irisknown <- iris[,5]
irisknown[134:150] <- NA
triris <- teigen(iris[,-5], models="CUUU", init="uniform", known=irisknown)

##Parallel examples:
###Note: parallel.cores set to 2 in order to comply
###with CRAN submission policies (set to higher
###number or TRUE to automatically use all available cores)

#Clustering old faithful data with tEIGEN
tfaith <- teigen(faithful, models="UUUU",
parallel.cores=2, Gs=1:3, init="hard")
plot(tfaith, what = "contour")

#Classification with the iris data set 
irisknown <- iris[,5]
irisknown[sample(1:nrow(iris),50)] <- NA
tiris <- teigen(iris[,-5], parallel.cores=2, models="CUUU",
init="uniform", known=irisknown)
tiris$tab