mnimput {mtsdi}

Multivariate Normal Imputation

Description

Perform the modified EM algorithm imputation on a normal multivariate dataset

Usage

mnimput(formula, dataset, by = NULL, log = FALSE, log.offset = 1, 
		eps = 1e-3, maxit = 1e2, ts = TRUE, method = "spline", 
		sp.control = list(df = NULL, weights = NULL), ar.control = 
		list(order = NULL, period = NULL), ga.control = list(formula, 
		weights = NULL), f.eps = 1e-6, f.maxit = 1e3, ga.bf.eps = 1e-6, 
		ga.bf.maxit = 1e3, verbose = FALSE, digits = getOption("digits"))

Arguments

formula

formula indicating the missing data frame, for instance, ~X1+X2+X3+...+Xp

dataset

data with missing values to be imputated

by

factor for variance windows. Default is NULL for a single variance matrix

log

logical. If TRUE data will be transformed into log scale. Default is FALSE

log.offset

If log is TRUE, log values will be shifted by this offset. Default is 1

eps

stop criterion

maxit

maximum number of iterations

ts

logical. TRUE if is time series

method

method for univariate time series filtering. It may be smooth, gam or arima. See Details

sp.control

list for Spline smooth control. See Details

ar.control

list for ARIMA fitting control. See Details

ga.control

list for GAM fitting control. See Details

f.eps

convergence criterion for the ARIMA filter. See arima

f.maxit

maximum number of iterations for the ARIMA filter. See arima

ga.bf.eps

covergence criterion for the backfitting algorithm of GAM models. See gam

ga.bf.maxit

maximum number of iterations for the backfitting algorithm of GAM models. See gam

verbose

if TRUE convergence information on each iteration is printed. Default is FALSE

digits

an integer indicating the decimal places. If not supplied, it is taken from options

Details

This is a modified version of the EM algorithm for imputation of missing values. It is also applicable to time series data. When it is explicited the time series attribute through the argument ts, missing values are estimated accounting for both correlation between time series and time structure of the series itself. Several filters can be used for prediction of the mean vector in the E-step.

One can select the method for the univariate time series filtering by the argument method. The default method is "spline". In this case a smooth spline is fitted to each of the time series at each iteration. Some parameters can be passed to smooth.spline through sm.control. df is a vector as long as the number of columns in dataset holding fixed degrees of freedom of the splines. If NULL, the degrees of freedom of each spline are chosen by cross-validation. If df has length 1, this values is recycled for all the covariates. weights must be a matrix of the same size of dataset with the weights for smooth.spline. If NULL, all the observations will have weights equal to 1.

Other possibity for time series filtering is to fitting an ARIMA model for each of the time series by setting method to "arima". The ARIMA models must be identified before using this function, nonetheless. arima function can be partially controlled through ar.control. Each column of order must hold the corresponding (p,d,q) parameters for each univariate time series if period is NULL. If period is not NULL, order must also hold the multiplicative seasonality parameters, so each column of order takes the form (p,d,q,P,D,Q). period is the multiplicative seasonality period. f.eps and f.maxit control de convergence of the ARIMA fitting algorithm. Convergence problems due non stationarity may arise when using this option.

Last but not least, a very interesting approach to modelling temporal patterns to use a full fledged regression model. It is possible to use generalised aditive (or linear) models with exogenous variates to proper filtering of time patterns. One must set method to gam and supply a vector of formulas in ga.control. One must supply one formula for each covariate. Using covariates that are part of the formula of the imputation model may yield some colinearity among the variates. See gam and glm for details. In order to use regression models for the level, set method to "gam"

Simulations have shown that the algorithm is stable and yields good results on imputation of normal data.

Value

The function returns an object of class mtsdi containing

call

function call

dataset

imputed dataset

muhat

estimated mean vector

sigmahat

estimated covariance matrix

missings

vector holding the number of missing values on each row

iterations

number of iterations until convergence or reach maxit

convergence

convergence value. See Details

converged

a logical indicating if the algorithm converged

time

elapsed time of the process

Author(s)

Washington Junger [e-mail] and Antonio Ponce de Leon [e-mail]

References

Junger, W.L. and Ponce de Leon, A. (2015) Imputation of Missing Data in Time Series for Air Pollutants. Atmospheric Environment, 102, 96-104.

Johnson, R., Wichern, D. (1998) Applied Multivariate Statistical Analysis. Prentice Hall.

Dempster, A., Laird, N., Rubin, D. (1977) Maximum Likelihood from Incomplete Data via the Algorithm EM. Journal of the Royal Statistical Society 39(B)), 1–38.

McLachlan, G. J., Krishnan, T. (1997) The EM algorithm and extensions. John Wiley and Sons.

Box, G., Jenkins, G., Reinsel, G. (1994) Time Series Analysis: Forecasting and Control. 3 ed. Prentice Hall.

Hastie, T. J.; Tibshirani, R. J. (1990) Generalized Additive Models. Chapman and Hall.

See Also

mnimput, predict.mtsdi, edaprep

Examples

data(miss)
f <- ~c31+c32+c33+c34+c35
## one-window covariance
i <- mnimput(f,miss,eps=1e-3,ts=TRUE, method="spline",sp.control=list(df=c(7,7,7,7,7)))
summary(i)

## two-window covariances
b<-c(rep("year1",12),rep("year2",12))
ii <- mnimput(f,miss,by=b,eps=1e-3,ts=TRUE, method="spline",sp.control=list(df=c(7,7,7,7,7)))
summary(ii)