IRTest_Mix {IRTest}

Item and ability parameters estimation for a mixed-format item response data

Description

This function estimates IRT item and ability parameters when a test consists of mixed-format items (i.e., a combination of dichotomous and polytomous items). In educational context, the combination of these two item formats takes an advantage; Dichotomous item format expedites scoring and is conducive to cover broad domain, while Polytomous item format (e.g., free response item) encourages students to exert complex cognitive skills (Lee et al., 2020). Based on Bock & Aitkin's (1981) marginal maximum likelihood and EM algorithm (EM-MML), this function incorporates several latent distribution estimation algorithms which could free the normality assumption on the latent variable. If the normality assumption is violated, application of these latent distribution estimation methods could reflect some features of the unknown true latent distribution, and, thus, could provide more accurate parameter estimates (Li, 2021; Woods & Lin, 2009; Woods & Thissen, 2006).

Usage

IRTest_Mix(
  data_D,
  data_P,
  model_D = "2PL",
  model_P = "GPCM",
  range = c(-6, 6),
  q = 121,
  initialitem_D = NULL,
  initialitem_P = NULL,
  ability_method = "EAP",
  latent_dist = "Normal",
  max_iter = 200,
  threshold = 1e-04,
  bandwidth = "SJ-ste",
  h = NULL
)

Arguments

data_D

A matrix or data frame of item responses where responses are coded as 0 or 1. Rows and columns indicate examinees and items, respectively.

data_P

A matrix or data frame of item responses coded as 0, 1, ..., m for the m+1 category item. Rows and columns indicate examinees and items, respectively.

model_D

A scalar or vector that represents types of item characteristic functions. Insert 1, "1PL", "Rasch", or "RASCH" for one-parameter logistic model, 2, "2PL" for two-parameter logistic model, and 3, "3PL" for three-parameter logistic model. The default is "2PL".

model_P

A character value for an IRT model to be applied. Currently, PCM, GPCM, and GRM are available. The default is "GPCM".

range

Range of the latent variable to be considered in the quadrature scheme. The default is from -6 to 6: c(-6, 6).

q

A numeric value that represents the number of quadrature points. The default value is 121.

initialitem_D

A matrix of initial item parameter values for starting the estimation algorithm. The default value is NULL.

initialitem_P

A matrix of initial item parameter values for starting the estimation algorithm. The default value is NULL.

ability_method

The ability parameter estimation method. The available options are Expected a posteriori (EAP) and Maximum Likelihood Estimates (MLE). The default is EAP.

latent_dist

A character string that determines latent distribution estimation method. Insert "Normal", "normal", or "N" for the normality assumption on the latent distribution, "EHM" for empirical histogram method (Mislevy, 1984; Mislevy & Bock, 1985), "2NM" or "Mixture" for using two-component Gaussian mixture distribution (Li, 2021; Mislevy, 1984), "DC" or "Davidian" for Davidian-curve method (Woods & Lin, 2009), "KDE" for kernel density estimation method (Li, 2022), and "LLS" for log-linear smoothing method (Casabianca & Lewis, 2015). The default value is set to "Normal" to follow the convention.

max_iter

A numeric value that determines the maximum number of iterations in the EM-MML. The default value is 200.

threshold

A numeric value that determines the threshold of EM-MML convergence. A maximum item parameter change is monitored and compared with the threshold. The default value is 0.0001.

bandwidth

A character value that can be used if latent_dist = "KDE". This argument determines the bandwidth estimation method for "KDE". The default value is "SJ-ste". See density for available options.

h

A natural number less than or equal to 10 if latent_dist = "DC" or "LLS". This argument determines the complexity of the distribution.

Details

Dichotomous: the probabilities for a correct response (u=1)

1) One-parameter logistic (1PL) model

P(u=1|\theta, b)=\frac{\exp{(\theta-b)}}{1+\exp{(\theta-b)}}

2) Two-parameter logistic (2PL) model

P(u=1|\theta, a, b)=\frac{\exp{(a(\theta-b))}}{1+\exp{(a(\theta-b))}}

3) Three-parameter logistic (3PL) model

P(u=1|\theta, a, b, c)=c + (1-c)\frac{\exp{(a(\theta-b))}}{1+\exp{(a(\theta-b))}}

Polytomous: the probability for scoring u=k (i.e., k=0, 1, ..., m; m \ge 2)

1) Partial credit model (PCM)

P(u=0|\theta, b_1, ..., b_{m})=\frac{1}{1+\sum_{c=1}^{m}{\exp{\left[\sum_{v=1}^{c}{a(\theta-b_v)}\right]}}}

P(u=1|\theta, b_1, ..., b_{m})=\frac{\exp{(\theta-b_1)}}{1+\sum_{c=1}^{m}{\exp{\left[\sum_{v=1}^{c}{\theta-b_v}\right]}}}

\vdots

P(u=m|\theta, b_1, ..., b_{m})=\frac{\exp{\left[\sum_{v=1}^{m}{\theta-b_v}\right]}}{1+\sum_{c=1}^{m}{\exp{\left[\sum_{v=1}^{c}{\theta-b_v}\right]}}}

2) Generalized partial credit model (GPCM)

P(u=0|\theta, a, b_1, ..., b_{m})=\frac{1}{1+\sum_{c=1}^{m}{\exp{\left[\sum_{v=1}^{c}{a(\theta-b_v)}\right]}}}

P(u=1|\theta, a, b_1, ..., b_{m})=\frac{\exp{(a(\theta-b_1))}}{1+\sum_{c=1}^{m}{\exp{\left[\sum_{v=1}^{c}{a(\theta-b_v)}\right]}}}

\vdots

P(u=m|\theta, a, b_1, ..., b_{m})=\frac{\exp{\left[\sum_{v=1}^{m}{a(\theta-b_v)}\right]}}{1+\sum_{c=1}^{m}{\exp{\left[\sum_{v=1}^{c}{a(\theta-b_v)}\right]}}}

3) Graded response model (GRM)

P(u=0|\theta, a, b_1, ..., b_{m})=1-\frac{1}{1+\exp{\left[-a(\theta-b_1)\right]}}

P(u=1|\theta, a, b_1, ..., b_{m})=\frac{1}{1+\exp{\left[-a(\theta-b_1)\right]}}-\frac{1}{1+\exp{\left[-a(\theta-b_2)\right]}}

\vdots

P(u=m|\theta, a, b_1, ..., b_{m})=\frac{1}{1+\exp{\left[-a(\theta-b_m)\right]}}-0

Latent distribution estimation methods

1) Empirical histogram method

P(\theta=X_k)=A(X_k)

where k=1, 2, ..., q, X_k is the location of the kth quadrature point, and A(X_k) is a value of probability mass function evaluated at X_k. Empirical histogram method thus has q-1 parameters.

2) Two-component Gaussian mixture distribution

P(\theta=X)=\pi \phi(X; \mu_1, \sigma_1)+(1-\pi) \phi(X; \mu_2, \sigma_2)

where \phi(X; \mu, \sigma) is the value of a Gaussian component with mean \mu and standard deviation \sigma evaluated at X.

3) Davidian curve method

P(\theta=X)=\left\{\sum_{\lambda=0}^{h}{{m}_{\lambda}{X}^{\lambda}}\right\}^{2}\phi(X; 0, 1)

where h corresponds to the argument h and determines the degree of the polynomial.

4) Kernel density estimation method

P(\theta=X)=\frac{1}{Nh}\sum_{j=1}^{N}{K\left(\frac{X-\theta_j}{h}\right)}

where N is the number of examinees, \theta_j is jth examinee's ability parameter, h is the bandwidth which corresponds to the argument bw, and K( \bullet ) is a kernel function. The Gaussian kernel is used in this function.

5) Log-linear smoothing method

P(\theta=X_{q})=\exp{\left(\beta_{0}+\sum_{m=1}^{h}{\beta_{m}X_{q}^{m}}\right)}

where h is the hyper parameter which determines the smoothness of the density, and \theta can take total Q finite values (X_1, \dots ,X_q, \dots, X_Q).

Value

This function returns a list of several objects:

par_est

The list of item parameter estimates. The first and second objects are the matrices of dichotomous and polytomous item parameter estimates, respectively

se

The list of standard errors of the item parameter estimates. The first and second objects are the matrices of standard errors of dichotomous and polytomous item parameter estimates, respectively

fk

The estimated frequencies of examinees at quadrature points.

iter

The number of EM-MML iterations elapsed for the convergence.

quad

The location of quadrature points.

diff

The final value of the monitored maximum item parameter change.

Ak

The estimated discrete latent distribution. It is discrete (i.e., probability mass function) by the quadrature scheme.

Pk

The posterior probabilities of examinees at quadrature points.

theta

The estimated ability parameter values. If ability_method = "MLE". If an examinee receives a maximum or minimum score for all items, the function returns \pmInf.

theta_se

Standard error of ability estimates. The asymptotic standard errors for ability_method = "MLE" (the function returns NA for all or none correct answers). The standard deviations of the posterior distributions for ability_method = "MLE".

logL

The deviance (i.e., -2logL).

density_par

The estimated density parameters.

Options

A replication of input arguments and other information.

Author(s)

Seewoo Li [e-mail]

References

Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46(4), 443-459.

Casabianca, J. M., & Lewis, C. (2015). IRT item parameter recovery with marginal maximum likelihood estimation using loglinear smoothing models. Journal of Educational and Behavioral Statistics, 40(6), 547-578.

Lee, W. C., Kim, S. Y., Choi, J., & Kang, Y. (2020). IRT Approaches to Modeling Scores on Mixed-Format Tests. Journal of Educational Measurement, 57(2), 230-254.

Li, S. (2021). Using a two-component normal mixture distribution as a latent distribution in estimating parameters of item response models. Journal of Educational Evaluation, 34(4), 759-789.

Li, S. (2022). The effect of estimating latent distribution using kernel density estimation method on the accuracy and efficiency of parameter estimation of item response models [Master's thesis, Yonsei University, Seoul]. Yonsei University Library.

Mislevy, R. J. (1984). Estimating latent distributions. Psychometrika, 49(3), 359-381.

Mislevy, R. J., & Bock, R. D. (1985). Implementation of the EM algorithm in the estimation of item parameters: The BILOG computer program. In D. J. Weiss (Ed.). Proceedings of the 1982 item response theory and computerized adaptive testing conference (pp. 189-202). University of Minnesota, Department of Psychology, Computerized Adaptive Testing Conference.

Woods, C. M., & Lin, N. (2009). Item response theory with estimation of the latent density using Davidian curves. Applied Psychological Measurement, 33(2), 102-117.

Woods, C. M., & Thissen, D. (2006). Item response theory with estimation of the latent population distribution using spline-based densities. Psychometrika, 71(2), 281-301.

Examples

# A preparation of mixed-format item response data

Alldata <- DataGeneration(N=1000,
                          nitem_D = 5,
                          nitem_P = 3)

DataD <- Alldata$data_D   # item response data for the dichotomous items
DataP <- Alldata$data_P   # item response data for the polytomous items

# Analysis

M1 <- IRTest_Mix(DataD, DataP)