detect_pm {aberrance} | R Documentation |
Detect parametric misfit
Description
Detect parametric misfit using person-fit statistics.
Usage
detect_pm(
method,
psi,
xi = NULL,
x = NULL,
d = NULL,
r = NULL,
y = NULL,
interval = c(-4, 4),
alpha = 0.05
)
Arguments
method |
The person-fit statistic(s) to compute. Options for score-based statistics are:
Options for distractor-based statistics are:
Options for score and distractor-based statistics are:
Options for response-based statistics are:
Options for response time-based statistics are:
Options for score and response time-based statistics are:
Options for response and response time-based statistics are:
Statistics ending in
|
psi |
A matrix of item parameters. |
xi |
A matrix of person parameters. If |
x , d , r , y |
Matrices of raw data. |
interval |
The interval to search for the person parameters. Default is
|
alpha |
Value(s) between 0 and 1 indicating the significance level(s)
used for flagging. Default is |
Value
A list is returned with the following elements:
stat |
A matrix of parametric person-fit statistics. |
pval |
A matrix of p-values. |
flag |
An array of flagging results. The first dimension corresponds to persons, the second dimension to methods, and the third dimension to significance levels. |
References
Bedrick, E. J. (1997). Approximating the conditional distribution of person fit indexes for checking the Rasch model. Psychometrika, 62(2), 191–199.
Drasgow, F., Levine, M. V., & Williams, E. A. (1985). Appropriateness measurement with polychotomous item response models and standardized indices. British Journal of Mathematical and Statistical Psychology, 38(1), 67–86.
Gorney, K. (2024). Three new corrections for standardized person-fit statistics for tests with polytomous items. British Journal of Mathematical and Statistical Psychology. Advance online publication.
Gorney, K., Sinharay, S., & Eckerly, C. (2024). Efficient corrections for standardized person-fit statistics. Psychometrika, 89(2), 569–591.
Gorney, K., Sinharay, S., & Liu, X. (2024). Using item scores and response times in person-fit assessment. British Journal of Mathematical and Statistical Psychology, 77(1), 151–168.
Gorney, K., & Wollack, J. A. (2023). Using item scores and distractors in person-fit assessment. Journal of Educational Measurement, 60(1), 3–27.
Molenaar, I. W., & Hoijtink, H. (1990). The many null distributions of person fit indices. Psychometrika, 55(1), 75–106.
Sinharay, S. (2016a). Asymptotic corrections of standardized extended caution indices. Applied Psychological Measurement, 40(6), 418–433.
Sinharay, S. (2016b). Asymptotically correct standardization of person-fit statistics beyond dichotomous items. Psychometrika, 81(4), 992–1013.
Sinharay, S. (2018a). A new person-fit statistic for the lognormal model for response times. Journal of Educational Measurement, 55(4), 457–476.
Sinharay, S. (2018b). Extension of caution indices to mixed-format tests. British Journal of Mathematical and Statistical Psychology, 71(2), 363–386.
Snijders, T. A. B. (2001). Asymptotic null distribution of person fit statistics with estimated person parameter. Psychometrika, 66(3), 331–342.
Tatsuoka, K. K. (1984). Caution indices based on item response theory. Psychometrika, 49(1), 95–110.
See Also
detect_nm()
to detect nonparametric misfit.
Examples
# Setup for Examples 1 and 2 ------------------------------------------------
# Settings
set.seed(0) # seed for reproducibility
N <- 500 # number of persons
n <- 40 # number of items
# Randomly select 10% examinees with preknowledge and 40% compromised items
cv <- sample(1:N, size = N * 0.10)
ci <- sample(1:n, size = n * 0.40)
# Create vector of indicators (1 = misfitting, 0 = fitting)
ind <- ifelse(1:N %in% cv, 1, 0)
# Example 1: Item Scores and Response Times ---------------------------------
# Generate person parameters for the 3PL model and lognormal model
xi <- MASS::mvrnorm(
N,
mu = c(theta = 0.00, tau = 0.00),
Sigma = matrix(c(1.00, 0.25, 0.25, 0.25), ncol = 2)
)
# Generate item parameters for the 3PL model and lognormal model
psi <- cbind(
a = rlnorm(n, meanlog = 0.00, sdlog = 0.25),
b = NA,
c = runif(n, min = 0.05, max = 0.30),
alpha = runif(n, min = 1.50, max = 2.50),
beta = NA
)
# Generate positively correlated difficulty and time intensity parameters
psi[, c("b", "beta")] <- MASS::mvrnorm(
n,
mu = c(b = 0.00, beta = 3.50),
Sigma = matrix(c(1.00, 0.20, 0.20, 0.15), ncol = 2)
)
# Simulate uncontaminated data
dat <- sim(psi, xi)
x <- dat$x
y <- dat$y
# Modify contaminated data by changing the item scores and reducing the log
# response times
x[cv, ci] <- rbinom(length(cv) * length(ci), size = 1, prob = 0.90)
y[cv, ci] <- y[cv, ci] * 0.75
# Detect parametric misfit
out <- detect_pm(
method = c("L_S_TS", "L_T", "Q_ST_TS", "L_ST_TS"),
psi = psi,
x = x,
y = y
)
# Example 2: Polytomous Item Scores -----------------------------------------
# Generate person parameters for the generalized partial credit model
xi <- cbind(theta = rnorm(N, mean = 0.00, sd = 1.00))
# Generate item parameters for the generalized partial credit model
psi <- cbind(
a = rlnorm(n, meanlog = 0.00, sdlog = 0.25),
c0 = 0,
c1 = rnorm(n, mean = -1.00, sd = 0.50),
c2 = rnorm(n, mean = 0.00, sd = 0.50),
c3 = rnorm(n, mean = 1.00, sd = 0.50)
)
# Simulate uncontaminated data
x <- sim(psi, xi)$x
# Modify contaminated data by changing the item scores to the maximum score
x[cv, ci] <- 3
# Detect parametric misfit
out <- detect_pm(
method = c("ECI2_S_TSCF", "ECI4_S_TSCF", "L_S_TSCF"),
psi = psi,
x = x
)