R: Choose the Next Item in a CAT

itChoose {catIrt}

R Documentation

Choose the Next Item in a CAT

Description

itChoose chooses the next item in a CAT based on the remaining items and a variety of item selection algorithms.

Usage

itChoose( left_par, mod = c("brm", "grm"), 
          numb = 1, n.select = 1,
          cat_par = NULL, cat_resp = NULL, cat_theta = NULL,
          select = c("UW-FI", "LW-FI", "PW-FI",
                     "FP-KL", "VP-KL", "FI-KL", "VI-KL",
                     "random"),
          at = c("theta", "bounds"),
          range = c(-6, 6), it.range = NULL,
          delta = NULL, bounds = NULL,
          ddist = dnorm, quad = 33, ... )

Arguments

`left_par`	numeric: a matrix of item parameters from which to choose the next item. The rows must index the items, and the columns must designate the item parameters (in the appropriate order, see `catIrt`). The first column of `left_par` must indicate the item numbers, as `itChoose` returns not just the item parameters but also the bank number corresponding to those parameters. See Details for more information.
`mod`	character: a character string indicating the IRT model. Current support is for the 3-parameter binary response model ("brm"), and Samejima's graded response model ("grm"). The contents of `params` must match the designation of `mod`. See `catIrt` or `simIrt` for more information.
`numb`	numeric: a scalar indicating the number of items to return to the user. If `numb` is less than `n.select`, then `itChoose` will randomly select `numb` items from the top `n.select` items according to the item selection algorithm.
`n.select`	numeric: an integer indicating the number of items to randomly select between at one time. For instance, if `select` is "UW-FI", `at` is "theta", `numb` is 3, and `n.select` is 8, then `itChoose` will randomly select 3 items out of the top 8 items that maximize Fisher information at `cat_theta`.
`cat_par`	numeric: either NULL or a matrix of item parameters that have already been administered in the CAT. `cat_par` only needs to be specified if letting `select` equal either "LW-FI", "PW-FI", "VP-KL" or "VI-KL". The format of `cat_par` must be the same as the format of `left_par`. See Details for more information.
`cat_resp`	numeric: either NULL or a vector of responses corresponding to the items specified in `cat_par`. `cat_par` only needs to be specified if letting `select` equal either "LW-FI" or "PW-FI".
`cat_theta`	numeric: either NULL or a scalar corresponding to the current ability estimate. `cat_theta` is not needed if selecting items at "bounds" or using "LW-FI" or "PW-FI" as the item selection algorithm.
`select`	character: a character string indicating the desired item selection method. Items can be selected either through maximum Fisher information or Kullback-Leibler divergence methods or randomly. The Fisher information methods include "UW-FI": unweighted Fisher information at a point. "LW-FI": Fisher information weighted across the likelihood function. "PW-FI": Fisher information weighted across the posterior distribution of `\theta`. And the Kullback-Leibler divergence methods include "FP-KL": pointwise KL divergence between [P +/- delta], where P is either the current `\theta` estimate or a classification bound. "VP-KL": pointwise KL divergence between [P +/- delta/sqrt(n)], where n is the number of items given to this point in the CAT. "FI-KL": KL divergence integrated along [P -/+ delta] with respect to P "VI-KL": KL divergence integrated along [P -/+ delta/sqrt(n)] with respect to P. See Details for more information.
`at`	character: a character string indicating where to select items.
`range`	numeric: a 2-element numeric vector indicating the range of values that `itChoose` should average over if `select` equals "LW-FI" or "PW-FI".
`it.range`	numeric: Either a 2-element numeric vector indicating the minimum and maximum allowed difficulty parameters for selected items (only if `mod` is equal to "brm") or NULL indicating no item parameter restrictions.
`delta`	numeric: a scalar indicating the multiplier used in item selection if a Kullback-Leibler method is chosen. For fixed-point KL divergence, `delta` is frequently .1 or .2, whereas in variable-point KL divergence, `delta` usually corresponds to 95 or 97.5 percentiles on a normal distribution.
`bounds`	numeric: a vector of fixed-points/bounds from which to select items if `at` equals "bounds".
`ddist`	function: a function indicating how to calculate prior densities if `select` equals "PW-FI" (i.e., weighting Fisher information on the posterior distribution). See `catIrt` for more information.
`quad`	numeric: a scalar indicating the number of quadrature points when `select` equals "LW-FI" or "PW-FI". See Details for more information.
`...`	arguments passed to `ddist`, usually distribution parameters identified by name.

Details

The function itChoose returns the next item(s) to administer in a CAT environment. The item selection algorithms fall into three major types: Fisher information, Kullback-Leibler divergence, and random.

If choosing items based on Fisher information (select equals "UW-FI", "LW-FI", or "PW-FI"), then items are selected based on some aggregation of Fisher information (see FI). The difference between the three Fisher information methods are the weighting functions used (see van der Linden, 1998; Veerkamp & Berger, 1997). Let

I(w_{ij} | a_j, b_j, c_j) = \int_{-\infty}^{\infty} w_{ij}I_j(\theta)\mu(d\theta)

be the "average" Fisher information, weighted by real valued function w_{ij}. Then all three Fisher information criteria can be explained solely as using different weights. Unweighted Fisher information ("UW-FI") sets w_{ij} equal to a Dirac delta function with all of its mass either on theta (if at equals "theta") or the nearest classification bound (if at equals "bounds"). Likelihood-Weighted Fisher information ("UW-FI") sets w_{ij} equal to the likelihood function given all of the previously administered items (Veerkamp & Berger, 1997). And Posterior-Weighted Fisher information ("PW-FI") sets w_{ij} equal to the likelihood function times the prior distribution specified in ddist (van der Linden, 1998). All three algorithms select items based on maximizing the respective criterion with "UW-FI" the most popular CAT item selection algorithm and equivalent to maximizing Fisher information at a point (Pashley & van der Linden, 2010).
If choosing items based on Kullback-Leibler divergence (select equals "FP-KL", "VP-KL", "FI-KL", or "VI-KL"), then items are selected based on some aggregation of KL divergence (see KL).
- The Pointwise KL divergence criteria (select equals "FP-KL" and "VP-KL") compares KL divergence at two points:
  
  KL(w_{ij} | a_j, b_j, c_j) = KL_j(P + w_{ij} || P - w_{ij})
  
  The difference between "FP-KL" and "VP-KL" are the weights used. Fixed Pointwise KL divergence ("FP-KL") sets w_{ij} equal to delta, and Variable Pointwise KL divergence ("VP-KL") sets w_{ij} equal to delta multiplied by 1/\sqrt{n}, where n is equal to the number of items given to this point in the CAT (see Chang & Ying, 1996).
- The Integral KL divergence criteria (select equals "FI-KL" and "VI-KL") integrates KL divergence across a small area:
  
  KL(w_{ij} | a_j, b_j, c_j) = \int_{P - w_{ij}}^{P + w_{ij}} KL_j(\theta || P) d\theta
  
  As in Pointwise KL divergence, Fixed Integral KL divergence ("FI-KL") sets w_{ij} equal to delta, and Variable Integral KL divergence ("VI-KL") sets w_{ij} equal to delta multiplied by 1/\sqrt{n} (see Chang & Ying, 1996).
All KL divergence criteria set P equal to theta (if at equals "theta") or the nearest classification bound (if at equals "bounds") and select items based on maximizing the respective criterion.
If select is "random", then itChoose randomly picks the next item(s) out of the remaining items in the bank.

Value

itChoose returns a list of the following elements:

`params`	a matrix corresponding to the next item or `numb` items to administer in a CAT with the first column indicating the item number
`info`	a vector of corresponding information for the `numb` items of `params`.
`type`	the type of information returned in `info`, which is equal to the item selection algorithm.

Author(s)

Steven W. Nydick swnydick@gmail.com

References

Chang, H.-H., & Ying, Z. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213 – 229.

Pashley, P. J., & van der Linden, W. J. (2010). Item selection and ability estimation in adaptive testing. In W. J. van der Linden & C. A. W. Glas (Eds.), Elements of adaptive testing (pp. 3 – 30). New York, NY: Springer.

van der Linden, W. J. (1998). Bayesian item selection criteria for adaptive testing. Psychometrika, 63, 201 – 216.

Veerkamp, W. J. J., & Berger, M. P. F. (1997). Some new item selection criteria for adaptive testing. Journal of Educational and Behavioral Statistics, 22, 203 – 226.

Examples

#########################
# Binary Response Model #
#########################
## Not run: 
set.seed(888)
# generating an item bank under a binary response model:
b.params <- cbind(a = runif(100, .5, 1.5), b = rnorm(100, 0, 2), c = .2)
# simulating responses using default theta:
b.mod <- simIrt(theta = 0, params = b.params, mod = "brm")

# separating the items into "administered" and "not administered":
left_par <- b.mod$params[1:95, ]
cat_par <- b.mod$params[96:100, ]
cat_resp <- b.mod$resp[ , 96:100]

# running simIrt automatically adds the item numbers to the front!

# attempting each item selection algorithm (except random):
uwfi.it <- itChoose(left_par = left_par, mod = "brm",
                    numb = 1, n.select = 1,
                    cat_theta = 0,
                    select = "UW-FI",
                    at = "theta")
lwfi.it <- itChoose(left_par = left_par, mod = "brm",
                    numb = 1, n.select = 1,
                    cat_par = cat_par, cat_resp = cat_resp,
                    select = "LW-FI")
pwfi.it <- itChoose(left_par = left_par, mod = "brm",
                    numb = 1, n.select = 1,
                    cat_par = cat_par, cat_resp = cat_resp,
                    select = "PW-FI", ddist = dnorm, mean = 0, sd = 1)

fpkl.it <- itChoose(left_par = left_par, mod = "brm",
                    numb = 1, n.select = 1,
                    cat_theta = 0,
                    select = "FP-KL",
                    at = "theta", delta = 1.96)
vpkl.it <- itChoose(left_par = left_par, mod = "brm",
                    numb = 1, n.select = 1,
                    cat_par = cat_par, cat_theta = 0,
                    select = "VP-KL",
                    at = "theta", delta = 1.96)
fikl.it <- itChoose(left_par = left_par, mod = "brm",
                    numb = 1, n.select = 1,
                    cat_theta = 0,
                    select = "FI-KL",
                    at = "theta", delta = 1.96)
vikl.it <- itChoose(left_par = left_par, mod = "brm",
                    numb = 1, n.select = 1,
                    cat_par = cat_par, cat_theta = 0,
                    select = "VI-KL",
                    at = "theta", delta = 1.96)

# which items were the most popular?
uwfi.it$params  # 61 (b close to 0)
lwfi.it$params  # 55 (b close to -2.5)
pwfi.it$params  # 16 (b close to -0.5)
fpkl.it$params  # 61 (b close to 0)
vpkl.it$params  # 61 (b close to 0)
fikl.it$params  # 16 (b close to -0.5)
vikl.it$params  # 16 (b close to -0.5)

# if we pick the top 10 items for "FI-KL":
fikl.it2 <- itChoose(left_par = left_par, mod = "brm",
                     numb = 10, n.select = 10,
                     cat_theta = 0,
                     select = "FI-KL",
                     at = "theta", delta = 1.96)

# we find that item 61 is the third best item
fikl.it2$params

# why did "LW-FI" pick an item with a strange difficulty?
cat_resp

# because cat_resp is mostly 0 ...
# --> so the likelihood is weighted toward negative numbers.

#########################
# Graded Response Model #
#########################
set.seed(999)
# generating an item bank under a graded response model:
g.params <- cbind(runif(100, .5, 1.5), rnorm(100), rnorm(100),
                                       rnorm(100), rnorm(100), rnorm(100))
# simulating responses (so that the parameters are ordered - see simIrt)
left_par <- simIrt(theta = 0, params = g.params, mod = "grm")$params

# now we can choose the best item for theta = 0 according to FI:
uwfi.it2 <- itChoose(left_par = left_par, mod = "brm",
                     numb = 1, n.select = 1,
                     cat_theta = 0,
                     select = "UW-FI",
                     at = "theta")
uwfi.it2

## End(Not run)

[Package catIrt version 0.5.1 Index]