itChoose {catIrt}R Documentation

Choose the Next Item in a CAT

Description

itChoose chooses the next item in a CAT based on the remaining items and a variety of item selection algorithms.

Usage

itChoose( left_par, mod = c("brm", "grm"), 
          numb = 1, n.select = 1,
          cat_par = NULL, cat_resp = NULL, cat_theta = NULL,
          select = c("UW-FI", "LW-FI", "PW-FI",
                     "FP-KL", "VP-KL", "FI-KL", "VI-KL",
                     "random"),
          at = c("theta", "bounds"),
          range = c(-6, 6), it.range = NULL,
          delta = NULL, bounds = NULL,
          ddist = dnorm, quad = 33, ... )

Arguments

left_par

numeric: a matrix of item parameters from which to choose the next item. The rows must index the items, and the columns must designate the item parameters (in the appropriate order, see catIrt). The first column of left_par must indicate the item numbers, as itChoose returns not just the item parameters but also the bank number corresponding to those parameters. See Details for more information.

mod

character: a character string indicating the IRT model. Current support is for the 3-parameter binary response model ("brm"), and Samejima's graded response model ("grm"). The contents of params must match the designation of mod. See catIrt or simIrt for more information.

numb

numeric: a scalar indicating the number of items to return to the user. If numb is less than n.select, then itChoose will randomly select numb items from the top n.select items according to the item selection algorithm.

n.select

numeric: an integer indicating the number of items to randomly select between at one time. For instance, if select is "UW-FI", at is "theta", numb is 3, and n.select is 8, then itChoose will randomly select 3 items out of the top 8 items that maximize Fisher information at cat_theta.

cat_par

numeric: either NULL or a matrix of item parameters that have already been administered in the CAT. cat_par only needs to be specified if letting select equal either "LW-FI", "PW-FI", "VP-KL" or "VI-KL". The format of cat_par must be the same as the format of left_par. See Details for more information.

cat_resp

numeric: either NULL or a vector of responses corresponding to the items specified in cat_par. cat_par only needs to be specified if letting select equal either "LW-FI" or "PW-FI".

cat_theta

numeric: either NULL or a scalar corresponding to the current ability estimate. cat_theta is not needed if selecting items at "bounds" or using "LW-FI" or "PW-FI" as the item selection algorithm.

select

character: a character string indicating the desired item selection method. Items can be selected either through maximum Fisher information or Kullback-Leibler divergence methods or randomly. The Fisher information methods include

  • "UW-FI": unweighted Fisher information at a point.

  • "LW-FI": Fisher information weighted across the likelihood function.

  • "PW-FI": Fisher information weighted across the posterior distribution of θ.

And the Kullback-Leibler divergence methods include

  • "FP-KL": pointwise KL divergence between [P +/- delta], where P is either the current θ estimate or a classification bound.

  • "VP-KL": pointwise KL divergence between [P +/- delta/sqrt(n)], where n is the number of items given to this point in the CAT.

  • "FI-KL": KL divergence integrated along [P -/+ delta] with respect to P

  • "VI-KL": KL divergence integrated along [P -/+ delta/sqrt(n)] with respect to P.

See Details for more information.

at

character: a character string indicating where to select items.

range

numeric: a 2-element numeric vector indicating the range of values that itChoose should average over if select equals "LW-FI" or "PW-FI".

it.range

numeric: Either a 2-element numeric vector indicating the minimum and maximum allowed difficulty parameters for selected items (only if mod is equal to "brm") or NULL indicating no item parameter restrictions.

delta

numeric: a scalar indicating the multiplier used in item selection if a Kullback-Leibler method is chosen. For fixed-point KL divergence, delta is frequently .1 or .2, whereas in variable-point KL divergence, delta usually corresponds to 95 or 97.5 percentiles on a normal distribution.

bounds

numeric: a vector of fixed-points/bounds from which to select items if at equals "bounds".

ddist

function: a function indicating how to calculate prior densities if select equals "PW-FI" (i.e., weighting Fisher information on the posterior distribution). See catIrt for more information.

quad

numeric: a scalar indicating the number of quadrature points when select equals "LW-FI" or "PW-FI". See Details for more information.

...

arguments passed to ddist, usually distribution parameters identified by name.

Details

The function itChoose returns the next item(s) to administer in a CAT environment. The item selection algorithms fall into three major types: Fisher information, Kullback-Leibler divergence, and random.

Value

itChoose returns a list of the following elements:

params

a matrix corresponding to the next item or numb items to administer in a CAT with the first column indicating the item number

info

a vector of corresponding information for the numb items of params.

type

the type of information returned in info, which is equal to the item selection algorithm.

Author(s)

Steven W. Nydick nydic001@umn.edu

References

Chang, H.-H., & Ying, Z. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213 – 229.

Pashley, P. J., & van der Linden, W. J. (2010). Item selection and ability estimation in adaptive testing. In W. J. van der Linden & C. A. W. Glas (Eds.), Elements of adaptive testing (pp. 3 – 30). New York, NY: Springer.

van der Linden, W. J. (1998). Bayesian item selection criteria for adaptive testing. Psychometrika, 63, 201 – 216.

Veerkamp, W. J. J., & Berger, M. P. F. (1997). Some new item selection criteria for adaptive testing. Journal of Educational and Behavioral Statistics, 22, 203 – 226.

See Also

catIrt, FI, KL, mleEst, simIrt

Examples

#########################
# Binary Response Model #
#########################
## Not run: 
set.seed(888)
# generating an item bank under a binary response model:
b.params <- cbind(a = runif(100, .5, 1.5), b = rnorm(100, 0, 2), c = .2)
# simulating responses using default theta:
b.mod <- simIrt(theta = 0, params = b.params, mod = "brm")

# separating the items into "administered" and "not administered":
left_par <- b.mod$params[1:95, ]
cat_par <- b.mod$params[96:100, ]
cat_resp <- b.mod$resp[ , 96:100]

# running simIrt automatically adds the item numbers to the front!

# attempting each item selection algorithm (except random):
uwfi.it <- itChoose(left_par = left_par, mod = "brm",
                    numb = 1, n.select = 1,
                    cat_theta = 0,
                    select = "UW-FI",
                    at = "theta")
lwfi.it <- itChoose(left_par = left_par, mod = "brm",
                    numb = 1, n.select = 1,
                    cat_par = cat_par, cat_resp = cat_resp,
                    select = "LW-FI")
pwfi.it <- itChoose(left_par = left_par, mod = "brm",
                    numb = 1, n.select = 1,
                    cat_par = cat_par, cat_resp = cat_resp,
                    select = "PW-FI", ddist = dnorm, mean = 0, sd = 1)

fpkl.it <- itChoose(left_par = left_par, mod = "brm",
                    numb = 1, n.select = 1,
                    cat_theta = 0,
                    select = "FP-KL",
                    at = "theta", delta = 1.96)
vpkl.it <- itChoose(left_par = left_par, mod = "brm",
                    numb = 1, n.select = 1,
                    cat_par = cat_par, cat_theta = 0,
                    select = "VP-KL",
                    at = "theta", delta = 1.96)
fikl.it <- itChoose(left_par = left_par, mod = "brm",
                    numb = 1, n.select = 1,
                    cat_theta = 0,
                    select = "FI-KL",
                    at = "theta", delta = 1.96)
vikl.it <- itChoose(left_par = left_par, mod = "brm",
                    numb = 1, n.select = 1,
                    cat_par = cat_par, cat_theta = 0,
                    select = "VI-KL",
                    at = "theta", delta = 1.96)

# which items were the most popular?
uwfi.it$params  # 61 (b close to 0)
lwfi.it$params  # 55 (b close to -2.5)
pwfi.it$params  # 16 (b close to -0.5)
fpkl.it$params  # 61 (b close to 0)
vpkl.it$params  # 61 (b close to 0)
fikl.it$params  # 16 (b close to -0.5)
vikl.it$params  # 16 (b close to -0.5)

# if we pick the top 10 items for "FI-KL":
fikl.it2 <- itChoose(left_par = left_par, mod = "brm",
                     numb = 10, n.select = 10,
                     cat_theta = 0,
                     select = "FI-KL",
                     at = "theta", delta = 1.96)

# we find that item 61 is the third best item
fikl.it2$params

# why did "LW-FI" pick an item with a strange difficulty?
cat_resp

# because cat_resp is mostly 0 ...
# --> so the likelihood is weighted toward negative numbers.

#########################
# Graded Response Model #
#########################
set.seed(999)
# generating an item bank under a graded response model:
g.params <- cbind(runif(100, .5, 1.5), rnorm(100), rnorm(100),
                                       rnorm(100), rnorm(100), rnorm(100))
# simulating responses (so that the parameters are ordered - see simIrt)
left_par <- simIrt(theta = 0, params = g.params, mod = "grm")$params

# now we can choose the best item for theta = 0 according to FI:
uwfi.it2 <- itChoose(left_par = left_par, mod = "brm",
                     numb = 1, n.select = 1,
                     cat_theta = 0,
                     select = "UW-FI",
                     at = "theta")
uwfi.it2

## End(Not run)


[Package catIrt version 0.5-0 Index]