KL {catIrt}  R Documentation 
KL
calculates the IRT implementation of KullbackLeibler
divergence for various IRT models given a vector of ability values,
a vector/matrix of item responses, an IRT model, and a value
indicating the halfwidth of an indifference region.
KL( params, theta, delta = .1 ) ## S3 method for class 'brm' KL( params, theta, delta = .1 ) ## S3 method for class 'grm' KL( params, theta, delta = .1 )
params 
numeric: a vector or matrix of item parameters. If specified
as a matrix, the rows must index the items, and the columns
must designate the item parameters. Furthermore, if calculating
expected information, the number of rows must match the number
of columns of 
theta 
numeric: a vector of ability values, one for each simulee. When performing
a classification CAT, 
delta 
numeric: a scalar or vector indicating the halfwidth of the indifference

The function KL
returns item divergence and test divergence for the binary
response model ("brm") and the graded response model ("grm"). KLdivergence
is defined as the following:
KL(θ_2  θ_1) = E_{θ_2}\log[L(θ_2)/L(θ_1)]
where L(θ) stands for the likelihood of θ. Essentially, KLdivergence is the expected loglikelihood gain when using the true model in place of an alternative model.
For the binary response model, KLdivergence for an item simplifies to the following:
KL_j(θ_2  θ_1)_j = p_j(θ_2)\log[p_j(θ_2)/p_j(θ_1)] + [1  p_j(θ_2)]\log[(1  p_j(θ_2))/(1  p_j(θ_1))]
where p_{ij} is the probability of response, as indicated in the help page for simIrt
For the graded response model, KLdivergence for an item simplifies to the following:
KL_j(θ_2  θ_1) = ∑_k{P_{jk}(θ_2)\log[P_{jk}(θ)2/P_{jk}(θ_1)]}
where P_{jk}(θ_2) is the probability of θ_2 responding in category k as
indicated in the help page for simIrt
. See Eggen (1999) as applied to classification
CAT and van der Linden and Pashley (2010) more generally.
Because of the properties of likelihood functions in item response models, test information is simply the sum of the item informations, or:
KL(θ_2  θ_1) = ∑_jKL_j(θ_2  θ_1)
KL
is frequently used to select items in a classification CAT where the hypotheses (e.g. being
in one category versus another category are well defined). If "being in the upper category" is
θ_2 and "being in the lower category" is θ_1, then θ_2 = B + δ
and θ_1 = B  δ where B is the boundary separating the lower category from the
upper category. Conversely, if using KL
to select items in a precision CAT, then
θ_2 = \hat{θ}_i + δ and θ_1 = \hat{θ}_i where \hat{θ}_i
is the current, best estimate of θ. See catIrt
for more information.
KL
, KL.brm
, and KL.grm
return a list of the following elements:
item 
either: (1) a N \times J matrix of item information for each simulee to
each item; (2) a Jlength vector of item information for one simulee to
each item; or (3) an Nlength vector of item information for all simulees
to one item, depending on the dimensions of 
test 
an Nlength vector of test information, one for each simulee. Test information is the sum of item information across items. See Details for more information. 
KullbackLeibler divergence in IRT is not true KL divergence, as the expectation
is with respect to a model that is not necessarily true. Furthermore, it is not reciprocal,
as KL(θ_1  θ_2) \neq KL(θ_2  θ_1). There have been
other KLbased item selection measures proposed, including global information. See
Eggen (1999) and itChoose
.
Steven W. Nydick nydic001@umn.edu
Eggen, T. J. H. M. (1999). Item selection in adaptive testing with the sequential probability ratio test. Applied Psychological Measurement, 23, 249 – 261.
Kullback, S., & Leibler, R. A. (1951). On information and sufficiency. The Annals of Mathematical Statistics, 22, 79 – 86.
van dr Linden, W. J. & Pashley, P. J. (2010). Item selection and ability estimation in adaptive testing. In W. J. van der Linden & C. A. W. Glas (Eds.), Elements of Adaptive Testing. New York, NY: Springer.
######################### # Binary Response Model # ######################### ## 1 ## set.seed(888) # generating random theta: theta < rnorm(20) # generating an item bank under a 3parameter binary response model: b.params < cbind(a = runif(100, .5, 1.5), b = rnorm(100, 0, 2), c = .2) # you can indicate class of params or extract it from simIrt object: class(b.params) < "brm" # calculating KL information with delta = .1: k.info1 < KL(params = b.params, theta = theta, delt = .1) # changing delta to .2 k.info2 < KL(params = b.params, theta = theta, delt = .2) # notice how the overall information has increased when increasing delt: k.info1$test; k.info2$test # also compare with Fisher information: f.info < FI(params = b.params, theta = theta, type = "expected") k.info2$test; f.info$test # Fisher information is much higher because of how it weighs things. ## 2 ## # we can maximize information at a boundary  say "0": k.info3 < KL(params = b.params, theta = 0, delta = .1) b.params[which.max(k.info3$item), ] # notice how the a parameter is high while the b parameter is close to # 0, so item selection is working. # does Fisher information choose a different item? f.info2 < FI(params = b.params, theta = 0, type = "expected") b.params[which.max(f.info2$item), ] # nope  although with more items, who knows? ######################### # Graded Response Model # ######################### ## 1 ## set.seed(999) # generating random theta theta < rnorm(20) # generating an item bank under a graded response model: g.params < cbind(runif(100, .5, 1.5), rnorm(100), rnorm(100), rnorm(100), rnorm(100), rnorm(100)) # simulating responses (so that the parameters are ordered  see simIrt) g.params < simIrt(theta = theta, params = g.params, mod = "grm")$params[ , 1] # we can calculate KL information as before, noting that class(g.params) is "grm" class(g.params) # so we don't need to set it ourselves # and now KL info with delt = .1 k.info4 < KL(theta = theta, params = g.params) # KL information is higher when more boundaries k.info4$test k.info1$test # Note: k.info1 would be exactly the same if calculated with the "grm" # rather than the "brm" ## 2 ## # we can also maximize information at boundary "0" k.info5 < KL(params = g.params, theta = 0, delta = .1) g.params[which.max(k.info5$item), ] # notice how the a parameter is high while the b parameters are pretty spread out. # does Fisher information choose a different item? f.info3 < FI(params = g.params, theta = 0, type = "expected") g.params[which.max(f.info3$item), ] # nope  although with more items, who knows?