gtheory_ml {quest}

Generalizability Theory Reliability of a Multilevel Score

Description

gtheory_ml uses generalizability theory to compute the reliability coefficients of a multilevel score. It computes a within-group coefficient that assesses the reliability of the group-deviated score (e.g., after calling center_by) and a between-group coefficient that assess the reliability of the mean aggregate score (e.g., after calling agg). It assumes two-level data where the rows are in long format and the columns are the variables/items of the score. Generaliability theory coefficients with multilevel data are analagous to intraclass correlations (ICC), but add an additional grouping variable. The default computes a multilevel version of ICC(3,k) from cross.obs = TRUE. When cross.obs = FALSE, a multilevel version of ICC(2,k) is computed, which takes mean differences between variables/items into account. gtheory_ml is a wrapper function for mlr. Note, this function can take several minutes to run if you have a moderate to large dataset.

Usage

gtheory_ml(data, vrb.nm, grp.nm, obs.nm, cross.obs = TRUE)

Arguments

data

data.frame of data.

vrb.nm

character vector of colnames from data specifying the variables/items.

grp.nm

character vector of length 1 with colname from data specifying the grouping variable. Because gtheory_ml is specific to two-level data, this can only be one variable.

obs.nm

character vector of of length 1 with colname from data specifying the observation variable. In this context, observation refers to comparable cases across groups. In a longitudinal study, the groups are people and the observations are timepoints. For example, each person has a timepoint 1, timepoint 2, timepoint 3, etc. In an school study, the groups are classrooms and the observations are students. For example, each classroom has a student 1, student 2, student 3, etc. While longitudinal studies often have a time variable in their data, school studies don't always have a student variable. You would then have to create a student variable to be able to use gtheory_ml.

cross.obs

logical vector of length 1 specifying whether the observations should be crossed when computing the generalizability theory coefficient. If TRUE, the observations are treated as fixed; if FALSE, they are treated as random. See details.

Details

gtheory_ml uses mlr, which is based on the formulas in Shrout, Patrick, and Lane (2012). When cross.obs = TRUE, the within-group coefficient is Rc and the between-group coefficient is RkF. When cross.obs = FALSE, the within-group coefficient is Rcn and the between-group coefficient is RkRn.

gtheory_ml does not currently have standard errors or confidence intervals. I am not aware of mathematical formulas for analytical confidence intervals, and because the generaliability theory coefficients can take several minutes to estimate, bootstraped confidence intervals seem too time-intensive to be useful at the moment.

gtheory_ml does not work with a single variable/item. You can still use generalizability theory to estimate between-group reliability in that instance though. To do so, reshape the variable/item from long to wide (e.g., unstack2) so that you have a column for each observation of that single variable/item and the rows are the groups. Then you can use gtheory and treat each observation as a "different" variable/item.

Value

list with two elements. The first is named "within" and refers to the within-group reliability. The second is named "between" and refers to the between-group reliability. Each contains a double vector where the first element is named "est" and contains the generalizability theory coefficient itself. The second element is named "average_r" and contains the average correlation at that level of the data based on cor_ml (which is a wrapper for statsBy). The third element is named "nvrb" and contains the number of variables/items. These later two elements are included because even though the reliability coefficients are calculated from variance components, they are indirectly based on the average correlation and number of variables/items, similar to Cronbach's alpha.

References

Shrout, Patrick and Lane, Sean P (2012), Psychometrics. In M.R. Mehl and T.S. Conner (eds) Handbook of research methods for studying daily life, (p 302-320) New York. Guilford Press

See Also

gtheorys_ml gtheory

Examples

shrout <- structure(list(Person = c(1L, 2L, 3L, 4L, 5L, 1L, 2L, 3L, 4L,
   5L, 1L, 2L, 3L, 4L, 5L, 1L, 2L, 3L, 4L, 5L), Time = c(1L, 1L,
      1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 3L, 4L, 4L, 4L,
      4L, 4L), Item1 = c(2L, 3L, 6L, 3L, 7L, 3L, 5L, 6L, 3L, 8L, 4L,
         4L, 7L, 5L, 6L, 1L, 5L, 8L, 8L, 6L), Item2 = c(3L, 4L, 6L, 4L,
            8L, 3L, 7L, 7L, 5L, 8L, 2L, 6L, 8L, 6L, 7L, 3L, 9L, 9L, 7L, 8L
         ), Item3 = c(6L, 4L, 5L, 3L, 7L, 4L, 7L, 8L, 9L, 9L, 5L, 7L,
            9L, 7L, 8L, 4L, 7L, 9L, 9L, 6L)), .Names = c("Person", "Time",
               "Item1", "Item2", "Item3"), class = "data.frame", row.names = c(NA,
                  -20L))
mlr_obj <- psych::mlr(x = shrout, grp = "Person", Time = "Time",
   items = c("Item1", "Item2", "Item3"),
   alpha = FALSE, icc = FALSE, aov = FALSE, lmer = TRUE, lme = FALSE,
   long = FALSE, plot = FALSE)
gtheory_ml(data = shrout, vrb.nm = c("Item1", "Item2", "Item3"),
   grp.nm = "Person", obs.nm = "Time", cross.obs = TRUE) # crossed time
gtheory_ml(data = shrout, vrb.nm = c("Item1", "Item2", "Item3"),
   grp.nm = "Person", obs.nm = "Time", cross.obs = FALSE) # nested time