qip_index {irt}

Calculate Quality of Item Pool Index

Description

The QIP Index can take values between 0 and 1 and indicates an item pool’s level of efficiency. A value of 1 signifies an optimum item pool for that examinee group. If one adds redundant items to an item pool that cannot be used by the CAT algorithm, the QIP Index will not increase or will increase minimally. In this sense, the QIP Index is an indicator of the item pools’ deficiency, instead of redundancy. However, if an exposure control mechanism is within test specifications, the QIP index can measure whether the redundancy in the item pool supports the exposure control method. See Gonulates (2019) for details.

Note that this function will best work with Rasch or 1PL models. It will not work with polytomous items.

Usage

qip_index(cat_sim_output, summary_func = NULL, ...)

Arguments

cat_sim_output

This is a list object containing elements that are cat_output class.

summary_func

A string representing the function that will be applied to individual QIP values for a simulee. The default is NULL, where all QIP values of each administered item of a simulee will be returned. Other possible values are: "mean", "median", "min", "max". See examples for demonstrations.

...

Additional arguments that will be passed to the summary_func. For example, if summary_func = "quantile", probability of the 25th quantile can be specified using the argument prob = .25. See examples for demonstrations.

Since ... will be passed to sapply function, simplify = FALSE can be passed to function to get results as list elements.

Value

A vector or matrix of QIP values or the summary statistics of QIP values.

Author(s)

Emre Gonulates

References

Gönülateş, E. (2019). Quality of Item Pool (QIP) Index: A Novel Approach to Evaluating CAT Item Pool Adequacy. Educational and Psychological Measurement, 79(6), 1133–1155. <doi:10.1177/0013164419842215>

Examples

cd <- create_cat_design(ip = generate_ip(n = 30), next_item_rule = 'mfi',
                        termination_rule = 'max_item',
                        termination_par = list(max_item = 10))
cat_output <- cat_sim(true_ability = rnorm(10), cd = cd)

qip_index(cat_output)

# Return result as list elements
qip_index(cat_output, simplify = FALSE)

# Summarize QIP values:
qip_index(cat_output, summary_func = "mean")
qip_index(cat_output, summary_func = "median")
qip_index(cat_output, summary_func = "min")
qip_index(cat_output, summary_func = "max")
qip_index(cat_output, summary_func = "quantile", prob = .25)
qip_index(cat_output, summary_func = "quantile", prob = c(.25, .5, .75))

qip_index(cat_output, summary_func = "quantile", prob = c(.25, .5, .75),
          simplify = FALSE)