wdm {ata}

Automated Test Assembly via Weighted (positive) Deviations Method

Description

Ingests item metadata jointly with target test form constraints and uses the Weighted (positive) Deviations Method (WDM) to construct a test form based on the desired objectives.

Usage

wdm( ipool,
 id,
 constraints,
 first = NA,
 refine = TRUE,
 permutate = FALSE,
 tieselect = 1,
 verbose = TRUE,
 aprioriadd = NA,
 posthocadd = NA )

Arguments

ipool

Item by characteristic (property) metadata pool.

id

Name of unique item identifier.

constraints

Complex list object identifying the constraints to be applied in the ATA (see makeconstobj for guided process).

first

How should item selection start: id of the item to be selected first from the pool, NA (default) - select randomly from the pool.

refine

Should the final test form be refined against the remaining item pool? Default is TRUE.

permutate

Assemble test forms starting with each item sequentially (as many forms as items in pool) and define final test form based on eligible constraint compliant solutions; Default is FALSE (currently not available).

tieselect

How should tied items be resolved: 1 (default) - select the first item in the list of candidates (sensitive to data sorting); not applicable for situations with all categorical constraints only, 0 - randomly select candidate; not applicable for situations with all categorical constraints only

verbose

Should progress of wdm() be printed to the console? Default = TRUE.

aprioriadd

Force item addition (via IDs) to test before ATA, which affects item selection and constraint attainment success (currently not available).

posthocadd

Force item addition (via IDs) to test after ATA, which affects final form specifications (currently not available).

Value

A complex list object with test assembly specific estimates:

wde

Estimates of the computational steps deriving the positive weighted deviations and item selection.

evaluation

Final assembled test form additive properties across constraints.

considered

Estimates of the computational steps when refine = TRUE evaluating selected items and selecting replacements.

excluded

Items from pool excluded.

excluded_set

Item sets excluded. Only included if input constobj includes a set_id.

included

Items from pool included in new test form.

included_set

Item sets from pool included in new test form. Only included if input constobj includes a set_id.

initial_ids

Item sets from pool included in new test form.

initial_setids

Item sets from pool included in new test form. Only included if input constobj includes a set_id.

final_ids

Final item ids in the test form.

final_setids

Final set ids in the test form. Only included if input constobj includes a set_id.

Author(s)

Gulsah Gurkan ([e-mail]), Michael Chajewski ([e-mail])

References

Parshall, C. G., et al. (2002). Automated test assembly for online administration. In C. G. Parshall, J. A. Spray, J. C. Kalohn, & T. Davey, Practical considerations in computer based testing (pp.106-125). New York, NY: Springer-Verlag New York, Inc.

Sanders, P. F., & Verschoor, A. J. (1998). Parallel test construction using classical item parameters. Applied Psychological Measurement, 22, 212-223.

Swanson, L., & Stocking, M. L. (1993). A Model and heuristic for solving Very large item selection problems. Applied Psychological Measurement, 17, 151-166.

Examples

# Specifying constraints
constin <- list(
  nI = 5,                                     # Number of items on the future test
  nC = 4,                                     # Number of constraints to be satisfied
  nameC = c("Content_A","Content_B","p","iSx"), # Name of constraint; must be numeric and must
  # reflect variable name in input
  lowerC = c(2, 3, 3.00, 0.50),               # Lower bound total constraint value on ATA form
  upperC = c(2, 3, 3.50, 0.60),               # Upper bound total constraint value on ATA form
  wC = c(1, 1, 1, 1),                         # Constraint weight used for weighted sum of
                                              # (positive) deviations St
  set_id = NA                                 # Aggregation ID for units / sets 
)

# Running WDM example from Parshall et al. (2002)
testWDM <- wdm( ipool = metadata_example,
                id = "Item",
                constraints = constin,
                first = 2)

# Summary of results
summary(testWDM)