reliability_dms {multid}

Reliability calculation for difference score variable that is a difference between two mean variables calculated over upper-level units (e.g., sex differences across countries)

Description

Calculates reliability of difference score (Johns, 1981) based on two separate ICC2 values (Bliese, 2000), standard deviations of mean values over upper-level units, and correlations between the mean values across upper-level units.

Usage

reliability_dms(
  model = NULL,
  data = NULL,
  diff_var,
  diff_var_values,
  var,
  group_var
)

Arguments

model

Multilevel model fitted with lmer (default NULL)

data

Long format data frame (default NULL)

diff_var

Character string. A variable indicative of difference score components (two groups).

diff_var_values

Vector. Values of the component score groups in diff_var.

var

Character string. Name of the dependent variable or variable of which mean values are calculated.

group_var

Character string. Upper-level clustering unit.

Value

A vector including ICC2s (r11 and r22), SDs (sd1, sd2, and sd_d12), means (m1, m2, and m_d12), correlation between means (r12), and reliability of the mean difference variable.

References

Bliese, P. D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 349–381). Jossey-Bass.

Johns, G. (1981). Difference score measures of organizational behavior variables: A critique. Organizational Behavior and Human Performance, 27(3), 443–463. https://doi.org/10.1016/0030-5073(81)90033-7

Examples

set.seed(4317)
n2 <- 20
n1 <- 200
ri <- rnorm(n2, m = 0.5, sd = 0.2)
rs <- 0.5 * ri + rnorm(n2, m = 0.3, sd = 0.15)
d.list <- list()
for (i in 1:n2) {
  x <- rep(c(-0.5, 0.5), each = n1 / 2)
  y <- ri[i] + rs[i] * x + rnorm(n1)
  d.list[[i]] <- cbind(x, y, i)
}

d <- data.frame(do.call(rbind, d.list))
names(d) <- c("x", "y", "cntry")
reliability_dms(
  data = d, diff_var = "x",
  diff_var_values = c(-0.5, 0.5), var = "y", group_var = "cntry"
)