gtheorys_ml {quest}

Generalizability Theory Reliability of Multiple Multilevel Scores

Description

gtheorys_ml uses generalizability theory to compute the reliability coefficients of multiple multilevel score. It computes within-group coefficients that assess the reliability of the group-deviated scores (e.g., after calling centers_by) and between-group coefficients that assess the reliability of the mean aggregate scores (e.g., after calling aggs). It assumes two-level data where the rows are in long format and the columns are the variables/items of the score. Generaliability theory coefficients with multilevel data are analagous to intraclass correlations (ICC), but add an additional grouping variable. The default computes a multilevel version of ICC(3,k) from cross.obs = TRUE. When cross.obs = FALSE, a multilevel version of ICC(2,k) is computed, which takes mean differences between variables/items into account. gtheorys_ml is a wrapper function for mlr. Note, this function can take several minutes to run if you have a moderate to large dataset.

Usage

gtheorys_ml(data, vrb.nm.list, grp.nm, obs.nm, cross.obs = TRUE)

Arguments

data

data.frame of data.

vrb.nm.list

list of character vectors of colnames from data specifying the sets of variables/items.

grp.nm

character vector of length 1 with colname from data specifying the grouping variable. Because gtheorys_ml is specific to two-level data, this can only be one variable.

obs.nm

character vector of of length 1 with colname from data specifying the observation variable. In this context, observation refers to comparable cases across groups. In a longitudinal study, the groups are people and the observations are timepoints. For example, each person has a timepoint 1, timepoint 2, timepoint 3, etc. In an school study, the groups are classrooms and the observations are students. For example, each classroom has a student 1, student 2, student 3, etc. While longitudinal studies often have a time variable in their data, school studies don't have always a student variable. You would then have to create a student variable to be able to use this function.

cross.obs

logical vector of length 1 specifying whether the observations should be crossed when computing the generalizability theory coefficients. If TRUE, the observations are treated as fixed; if FALSE, they are treated as random. See details.

Details

gtheorys_ml uses mlr, which is based on the formulas in Shrout, Patrick, and Lane (2012). When cross.obs = TRUE, the within-group coefficient is Rc and the between-group coefficient is RkF. When cross.obs = FALSE, the within-group coefficient is Rcn and the between-group coefficient is RkRn.

gtheorys_ml does not currently have standard errors or confidence intervals. I am not aware of mathematical formulas for analytical confidence intervals, and because the generaliability theory coefficients can take several minutes to estimate, bootstraped confidence intervals seem too time-intensive to be useful at the moment.

gtheorys_ml does not work with multiple single variable/item scores. You can still use generalizability theory to estimate between-group reliability in that instance though. To do so, reshape the multiple single variables/items from long to wide (e.g., long2wide) so that you have a column for each observation of that single variable/item and the rows are the groups. Then you can use gtheorys and treat each observation as a "different" variable/item.

Value

list with two elements. The first is named "within" and refers to the within-group reliability. The second is named "between" and refers to the between-group reliability. Each contains a data.frame with the following columns:

est

generalizability theory reliability coefficient itself

average_r

the average correlation at each level of the data based on cor_ml (which is a wrapper for statsBy)

nvrb

number of variables/items that make up that score

The later two columns are included because even though the reliability coefficients are calculated from variance components, they are indirectly based on the average correlation and number of variables/items similar to Cronbach's alpha.

References

Shrout, Patrick and Lane, Sean P (2012), Psychometrics. In M.R. Mehl and T.S. Conner (eds) Handbook of research methods for studying daily life, (p 302-320) New York. Guilford Press

See Also

gtheory_ml gtheorys

Examples

dat <- psychTools::sai[psychTools::sai$"study" == "VALE", ] # 4 timepoints
vrb_nm_list <- list("positive_affect" = c("calm","secure","at.ease","rested",
   "comfortable","confident"), # extra: "relaxed","content","joyful"
   "negative_affect" = c("tense","regretful","upset","worrying","anxious",
      "nervous")) # extra: "jittery","high.strung","worried","rattled"
suppressMessages(gtheorys_ml(data = dat, vrb.nm.list = vrb_nm_list, grp.nm = "id",
   obs.nm = "time", cross.obs = TRUE))
suppressMessages(gtheorys_ml(data = dat, vrb.nm.list = vrb_nm_list, grp.nm = "id",
   obs.nm = "time", cross.obs = FALSE))
gtheorys_ml(data = dat, vrb.nm.list = vrb_nm_list["positive_affect"], grp.nm = "id",
   obs.nm = "time") # also works with only one set of variables/items