atalp {ata}R Documentation

Automated Test Assembly via Linear Constrained Programming


Ingests item metadata jointly with target test form constraints, and can be parametarized to uses either Boolean (0-1) Integer Linear Programming (ILP) or Mixed Integer Linear Programming (MILP) to construct a test form based on the desired objectives. When MILP is desired the selection of the objective function type should be changed.


atalp( ipool,
      refine = FALSE,
      permutate = FALSE,
      sorttimes = 999,
      tieselect = -1,
      type = "const",
      verbose = TRUE,
      aprioriadd = NA,
      posthocadd = NA )



Item by characteristic (property) metadata pool.


Name of unique item identifier.


Complex list object identifying the constraints to be applied in the ATA (see makeconstobj for guided process).


Creates a final test form from permutated solutions, refined to attempt a deviation balance between the observed form and the constraints. Option only effective if permutate is TRUE and type = const in which the constraint weights have meaning; Default is FALSE.


Requests the test form to be assembled by resorting (sorttimes) the metadata and selecting the most frequently occurring item combination satisfying the constraints. Relevant only for type = const; Default is FALSE.


Number of how often the original input metadata should be resorted. Only functional if permutate is TRUE and type = const; default 999, so that sorttimes + main analysis account for a total of 1,000 selection versions.


How should tied items be resolved: -1 (default) - do not manipulate items (which allows for identically functioning items to be included), 1 - select the first item in the list of candidates (sensitive to data sorting); not applicable for situations with all categorical constraints only, 0 - randomly select candidate; not applicable for situations with all categorical constraints only.


Type of objective function: const - constraint based only (default), parmin - constraint + minimum non-categorical parameter combination, parmax - constraint + maximum non-categorical parameter combination.


Should progress be printed to the console? Default TRUE.


Force item addition (via IDs) to test form before ATA, which affects item selection and constraint attainment success (currently not available).


Force item addition (via IDs) to test form after ATA, which affects final form specifications (currently not available).


A complex list object with test assembly specific estimates:


Constrained objective function value.


Removed items from item pool when tieselect is not -1.


Items from pool excluded.


Item sets excluded. Only included if input constobj includes a set_id.


Items from pool included in new test form.


Item sets from pool included in new test form. Only included if input constobj includes a set_id.


Final item ids in the test form.


Final set ids in the test form. Only included if input constobj includes a set_id.


Michael Chajewski (, Gulsah Gurkan (


Chen, P. (2017). Should we stop developing heuristics and only rely on mixed integer programming solvers in automated test assembly? Applied Psychological Measurement, 41, 227-240.

Diao, Q., & van der Linden, W. J. (2011). Automated test assembly using lp_Solve Version 5.5 in R. Applied Psychological Measurement, 35, 398-409.

Shao, C., Liu, S., Yang, H., & Tsai, T. (2019). Automated test assembly using SAS operations research software in a medical licensing examination. Applied Psychological Measurement, 00, 1-15.

van der Linden, W. J. (2005). A comparison of item-selection methods for adaptive tests with content constraints. Journal of Educational Measurement, 42, 283-302.


# Specifying constraints
constin <- list(
  nI = 5,                                     # Number of items on the future test
  nC = 4,                                     # Number of constraints to be satisfied
  nameC = c("Content_A","Content_B","p","iSx"), # Name of constraint; must be numeric and must
  # reflect variable name in input
  lowerC = c(2, 3, 3.00, 0.50),               # Lower bound total constraint value on ATA form
  upperC = c(2, 3, 3.50, 0.60),               # Upper bound total constraint value on ATA form
  wC = c(1, 1, 1, 1),                         # Constraint weight used for weighted sum of
                                              # (positive) deviations St
  set_id = NA                                 # Aggregation ID for units / sets 

# Running atalp
testLP <- atalp(ipool = metadata_example,
                id = "Item",
                constraints = constin)

# Summary of results

[Package ata version 1.1.1 Index]