randomMST {mstR}

Random generation of multistage tests (dichotomous and polytomous models)

Description

This command generates a response pattern to a multistage test, for a given item bank (with either dichotomous or polytomous models), an MST structure for modules and stages, a true ability level, and several lists of MST parameters.

Usage

randomMST(trueTheta, itemBank, modules, transMatrix, model = NULL, 
  responses = NULL, genSeed = NULL, start = list(fixModule = NULL, seed = NULL, 
  theta = 0, D = 1), test = list(method = "BM", priorDist = "norm", 
  priorPar = c(0, 1), range = c(-4, 4), D = 1, parInt = c(-4, 4, 33), 
  moduleSelect = "MFI", constantPatt = NULL, cutoff = NULL, randomesque = 1,
  random.seed = NULL, score.range = "all"), final = list(method = "BM", 
  priorDist = "norm", priorPar = c(0, 1), range = c(-4, 4), D = 1, 
  parInt = c(-4, 4, 33), alpha = 0.05), allTheta = FALSE, save.output = FALSE, 
  output = c("path", "name", "csv"))
## S3 method for class 'mst'
print(x, ...)
## S3 method for class 'mst'
plot(x, show.path = TRUE, border.col = "red", arrow.col = "red",
  module.names = NULL, save.plot = FALSE, save.options = c("path", "name", "pdf"),...)
 

Arguments

trueTheta

numeric: the value of the true ability level.

itemBank

numeric: a suitable matrix of item parameters. See Details.

modules

a binary matrix that specifies the item membership to the modules. See Details.

transMatrix

a binary squared matrix representing the structure of the MST and the transitions between the moduels and the stages. See Details.

model

either NULL (default) for dichotomous models, or any suitable acronym for polytomous models. Possible values are "GRM", "MGRM", "PCM", "GPCM", "RSM" and "NRM". See Details.

responses

either NULL (default) or a vector of pre-specified item responses with as many components as the rows of itemBank. See Details.

genSeed

either a numeric value to fix the random generation of responses pattern or NULL (default). Ignored if responses is not NULL. See Details.

start

a list with the options for starting the multistage test. See Details.

test

a list with the options for provisional ability estimation and next module selection. See Details.

final

a list with the options for final ability estimation or scoring. See Details.

allTheta

logical: should all provisional ability estimates and standard errors be computed and returned (even within each module)? Default is FALSE, meaning that provisional ability estimates and standard errors are computed only at the end of each module administration.

save.output

logical: should the output be saved in an external text file? (default is FALSE).

output

character: a vector of three components. The first component is either the file path to save the output of "path" (default), the second component is the name of the output file, and the third component is the file type, either "txt" or "csv" (default). See Details.

x

either an object of class "mst", typically an output of randomMST function, or a transition matrix (for plot.mst() function only).

show.path

logical: should the selected path (i.e. set of successive modules) be highlighted in the plot (default is TRUE)?

border.col

character: the color for the rectangle border of the path (i.e. selected modules). Default is "red". Ignored if show.path is FALSE.

arrow.col

character: the color for the connecting arrows in the path (i.e. between selected modules). Default is "red". Ignored if show.path is FALSE.

module.names

either NULL (default) or a vector of character names for the modules. See Details.

save.plot

logical: should the plot be saved in an external figure? (default is FALSE).

save.options

character: a vector of three components. The first component is either the file path or "path" (default), the second component is the name of the output file or ,"name" (default), and the third component is the file extension, either "pdf" (default) or "jpeg". Ignored if save.plot is FALSE. See Details.

...

other generic arguments to be passed to print and plot functions.

Details

The randomMST function generates a multistage test using an item bank specified by arguments itemBank and model, an MST structure provided by arguments modules and transMatrix, and for a given true ability level specified by argument trueTheta.

Dichotomous IRT models are considered whenever model is set to NULL (default value). In this case, itemBank must be a matrix with one row per item and four columns, with the values of the discrimination, the difficulty, the pseudo-guessing and the inattention parameters (in this order). These are the parameters of the four-parameter logistic (4PL) model (Barton and Lord, 1981). See genDichoMatrix for further information.

Polytomous IRT models are specified by their respective acronym: "GRM" for Graded Response Model, "MGRM" for Modified Graded Response Model, "PCM" for Partical Credit Model, "GPCM" for Generalized Partial Credit Model, "RSM" for Rating Scale Model and "NRM" for Nominal Response Model. The itemBank still holds one row per item, end the number of columns and their content depends on the model. See genPolyMatrix for further information and illustrative examples of suitable polytomous item banks.

The modules argument must be a binary 0/1 matrix with as many rows as the item bank itemBank and as many columns as the number of modules. Values of 1 indicate to which module(s) the items belong to, i.e. a value of 1 on row i and column j means that the i-th item belongs to the j-th module.

The transMatrix argument must be a binary 0/1 square matrix with as many rows (and columns) as the number of modules. All values of 1 indicate the possible transitions from one module to another, i.e. a value of 1 on row i and column j means that the MST can move from i-th module to j-th module.

By default all item responses will be randomly drawn from parent distribution set by the item bank parameters of the itemBank matrix (using the genPattern function for instance). Moreover, the random generation of the item responses can be fixed (for e.g., replication purposes) by assigning some numeric value to the genSeed argument. By default this argument is equal to NULL so the random seed is not fixed (and two successive runs of randomMST will usually lead to different response patterns).

It is possible, however, to provide a full response pattern of previously recorded responses to each item of the item bank, for instance for post-hoc simulations. This is done by providing to the responses argument a vector of binary entries (without missing values). By default responses is set to NULL and item responses will be drawn from the item bank parameters.

The test specification is made by means of three lists of options: one list for the selection of the starting module, one list with the options for provisional ability estimation and next module selection, and one list with the options for final ability estimation. These lists are specified respectively by the arguments start, test and final.

The start list can contain one or several of the following arguments:

• fixModule: either an integer value, setting the module to be administered as first stage (as its row number in te transMatrix argument for instance), or NULL (default) to let the function select the module.

• seed: either a numeric value to fix the random seed for module selection, NA to randomly select the module withour fixing the random seed, or NULL (default) to make random module selection without fixing the random seed. Ignored if fixModule is not NULL.

• theta: the initial ability value, used to select the most informative module at this ability level (default is 0). Ignored if either fixModule or seed is not NULL. See startModule for further details.

• D: numeric, the metric constant. Default is D=1 (for logistic metric); D=1.702 yields approximately the normal metric (Haley, 1952). Ignored if model is not NULL and if startSelect is not "MFI".

These arguments are passed to the function startModule to select the first module of the multistage test.

The test list can contain one or several of the following arguments:

• method: a character string to specify the method for ability estimation or scoring. Possible values are: "BM" (default) for Bayesian modal estimation (Birnbaum, 1969), "ML" for maximum likelihood estimation (Lord, 1980), "EAP" for expected a posteriori (EAP) estimation (Bock and Mislevy, 1982), and "WL" for weighted likelihood estimation (Warm, 1989). The method argument can also take the value "score", meaning that module selection is based on the test score from the previously administered modules. The latter works only if cutoff argument is supplied appropriately, otherwise this leads to an error message.

• priorDist: a character string which sets the prior distribution. Possible values are: "norm" (default) for normal distribution, "unif" for uniform distribution, and "Jeffreys" for Jeffreys' noninformative prior distribution (Jeffreys, 1939, 1946). ignored if method is neither "BM" nor "EAP".

• priorPar: a vector of two numeric components, which sets the parameters of the prior distribution. If (method="BM" or method=="EAP") and priorDist="norm", the components of priorPar are respectively the mean and the standard deviation of the prior normal density. If (method="BM" or method="EAP") and priorDist="unif", the components of priorPar are respectively the lower and upper bound of the prior uniform density. Ignored in all other cases. By default, priorPar takes the parameters of the prior standard normal distribution (i.e., priorPar=c(0,1)). In addition, priorPar also provides the prior parameters for the comoutation of MLWI and MPWI values for next item selection (see nextModule for further details).

• range: the maximal range of ability levels, set as a vector of two numeric components. The ability estimate will always lie to this interval (set by default to [-4, 4]). Ignored if method=="EAP".

• D: the value of the metric constant. Default is D=1 for logistic metric. Setting D=1.702 yields approximately the normal metric (Haley, 1952). Ignored if model is not NULL.

• parInt: a numeric vector of three components, holding respectively the values of the arguments lower, upper and nqp of the eapEst, eapSem and MWI commands. It specifies the range of quadrature points for numerical integration, and is used for computing the EAP estimate, its standard error, and the MLWI and MPWI values for next item selection. Default vector is (-4, 4, 33), thus setting the range from -4 to 4 by steps of 0.25. Ignored if method is not "EAP" and if itemSelect is neither "MLMWI" nor "MPMWI".

• moduleSelect: the rule for next module selecion, with possible values:

  1. "MFI" (default) for maximum Fisher information criterion;

  2. "MLMWI" for maximum likelihood weighted (module) information criterion;

  3. "MPMWI" for posterior weighted (module) information criterion;

  4. "MKL" for (module) Kullback-Leibler information methods;

  5. "MKLP" for posterior (module) Kullback-Leibler information methods;

  6. "random" for random selection.

  This argument is ignored if cutoff is supplied appropriately. See nextModule for further details.

• constantPatt: the method to estimate ability in case of constant pattern (i.e. only correct or only incorrect responses). Can be either NULL (default), "BM", "EAP", "WL", "fixed4" (for fixed stepsize adjustment with step 0.4), "fixed7" (for fixed stepsize adjustment with step 0.7), or "var" (for variable stepsize adjustment). This is currently implemented only for dichotomous IRT models and is sgnored if method is "score". See thetaEst for further details.

• cutoff: either NULL (default) or a suitable matrix of thresholds to select the next module. Thresholds can reflect module selection based on ability estimation (and then method should hold one of the ability estimation methods) or on provisional test score (and then method must be set to "score". See nextModule for further details about suitable definition of the cutoff matrix (and the examples below).

• randomesque: a probability value to select the optimal module. Default is one, so the optimal module is always chosen. With a value smaler than one, other elligible modules can be selected.

• random.seed: either NULL (default) or a numeric value to fix the random seed of randomesque selection of the module. Ignored if randomesque is equal to one.

• score.range: a character value that specifies on which set of modules the provisional test score should be computed. Possible values are "all" (default) to compute the score with all previously administered modules, or "last" to compute the score only with the last module. Ignored if method is not "score".

These arguments are passed to the functions thetaEst and semTheta to estimate the ability level and the standard error of this estimate. In addition, some arguments are passed to nextModule to select the next module appropriately.

Finally, the final list can contain the arguments method, priorDist, priorPar, range, D and parInt of the test list (with possiblly different values), as well as the additional alpha argument. The latter specifies the \alpha level of the final confidence interval of ability, which is computed as

[\hat{\theta}-z_{1-\alpha/2} \; se(\hat{\theta}) ; \hat{\theta}+z_{1-\alpha/2} \; se(\hat{\theta})]

where \hat{\theta} and se(\hat{\theta}) are respectively the ability estimate and its standard error.

If some arguments of these lists are missing, they are automatically set to their default value.

Usually the ability estimates and related standard errors are computed right after the full administration of each module (that is, if current module has k items, the (k-1) ability levels and standard errors from the first administered (k-1) are not computed). This can however be avoided by fixing the argument allTheta to TRUE (by default it is FALSE). In this case, all provisional ability estimates (or test scores) and standard errors (or NA's) are computed and returned.

The output of randomMST, as displayed by the print.mst function, can be stored in a text file provided that save.output is set to TRUE (the default value FALSE does not execute the storage). In this case, the (output argument mus hold three character values: the path to where the output file must be stored, the name of the output file, and the type of output file. If the path is not provided (i.e. left to its default value "path"), it will be saved in the default working directory. The default name is "name", and the default type is "csv". Any other value yields a text file. See the Examples section for an illustration.

The function plot.mst represents the whole MST structure with as many rectangles as there are available modules, arrows connecting all the modules according to the transMatrix structure. Each stage is displayed as one horizontal layout with stage 1 on the top and final stage at the bottom of the figure. The selected path (i.e. set of modules) is displayed on the plot when show.path is TRUE (which is the default value). Modules from the path and arrows between them are then highlighted in red (by default), and these colors can be modified by settong border.col and arrow.col arguments with appropriate color names. By default, modules are labelled as “module 1", “module 2" etc., the numbering starting from left module to right module and from stage 1 to last stage. These labels can be modified by providing a vector of character names to argument module.names. This vector must have as many components as the total number of modules and being ordered identically as described above.

Note that the MST structure can be graphically displayed by only providing (as x argument) the transition matrix of the MST. In this case, show.path argument is ignored. This is useful to represent the MST structure set by the transition matrix without running an MST simulation.

Finally, the plot can be saved in an external file, either as PDF or JPEG format. First, the argument save.plot must be set to TRUE (default is FALSE). Then, the file path for figure storage, the name of the figure and its format are specified through the argument save.options, all as character strings. See the Examples section for further information and a practical example.

Value

The function randomMST returns a list of class "mst" with the following arguments:

trueTheta

the value of the trueTheta argument.

selected.modules

a vector with the modules (identified by their position in the transition matrix) that were selected for the MST.

items.per.module

a vector with the number of items per selected module (in the same order as in selected.modules).

transMatrix

the value of the transMatrix argument.

model

the value of the model argument.

testItems

a vector with the items that were administered during the test.

itemPar

a matrix with the parameters of the items administered during the test.

pattern

the generated (or selected) response pattern (as vector of 0 and 1 entries for dichotomous items or positive integer values for polytomous items).

thetaProv

a vector with the provisional ability estimates (or test scores if test$method is "score").

seProv

a vector with the standard errors of the provisional ability estimates (or vector of NA's if test$method is "score").

thFinal

the final ability estimate (or test score if test$method is "score").

seFinal

the standard error of the final ability estimate (or NA if test$method is "score").

ciFinal

the confidence interval of the final ability estimate (or c(NA, NA) if test$method is "score").

genSeed

the value of the genSeed argument.

startFixModule

the value of the start$fixModule argument (or its default value if missing).

startSeed

the value of the start$seed argument (or its default value if missing).

startTheta

the value of the start$theta argument (or its default value if missing).

startD

the value of the start$D argument (or its default value if missing).

startThStart

the starting ability value used for selecting the first module of the test.

startSelect

the value of the start$startSelect argument (or its default value if missing).

provMethod

the value of the test$method argument (or its default value if missing).

provDist

the value of the test$priorDist argument (or its default value if missing).

provPar

the value of the test$priorPar argument (or its default value if missing).

provRange

the value of the test$range argument (or its default value if missing).

provD

the value of the test$D argument (or its default value if missing)or NA if model is not NULL.

moduleSelect

the value of the test$moduleSelect argument (or its default value if missing).

constantPattern

the value of the test$constantPatt argument (or its default value if missing).

cutoff

the value of the test$cutoff argument (or its default value if missing).

randomesque

the value of the test$randomesque argument (or its default value if missing).

random.seed

the value of the test$random.seed argument (or its default value if missing).

score.range

the value of the test$score.range argument (or its default value if missing).

best.module

a vector of boolean values indicating whether the optimal modules were selected or not.

finalMethod

the value of the final$method argument (or its default value if missing).

finalDist

the value of the final$priorDist argument (or its default value if missing).

finalPar

the value of the final$priorPar argument (or its default value if missing).

finalRange

the value of the final$range argument (or its default value if missing).

finalD

the value of the final$D argument (or its default value if missing), or NA if model is not NULL.

finalAlpha

the value of the final$alpha argument (or its default value if missing).

save.output

the value of the save.output argument.

output

the value of the output argument.

assigned.responses

a logical value, being TRUE if responses was provided or FALSE responses was set to NULL.

allTheta

either a table with all ad-interim ability estimates (even within module, in the CAT spirit) if allTheta is set to TRUE, or NULL if allTheta is set to FALSE.

assigned.responses

the value of the responses argument (or its default value if missing).

The function print.mst returns similar (but differently organized) results.

Author(s)

David Magis
Department of Psychology, University of Liege, Belgium
[e-mail]

Duanli Yan
Educational Testing Service, Princeton, USA
[e-mail]

References

Barton, M.A., and Lord, F.M. (1981). An upper asymptote for the three-parameter logistic item-response model. Research Bulletin 81-20. Princeton, NJ: Educational Testing Service.

Birnbaum, A. (1969). Statistical theory for logistic mental test models with a prior distribution of ability. Journal of Mathematical Psychology, 6, 258-276. doi: 10.1016/0022-2496(69)90005-4

Bock, R. D., and Mislevy, R. J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431-444. doi: 10.1177/014662168200600405

Haley, D.C. (1952). Estimation of the dosage mortality relationship when the dose is subject to error. Technical report no 15. Palo Alto, CA: Applied Mathematics and Statistics Laboratory, Stanford University.

Jeffreys, H. (1939). Theory of probability. Oxford, UK: Oxford University Press.

Jeffreys, H. (1946). An invariant form for the prior probability in estimation problems. Proceedings of the Royal Society of London. Series A, Mathematical and Physical Sciences, 186, 453-461.

Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.

Warm, T.A. (1989). Weighted likelihood estimation of ability in item response models. Psychometrika, 54, 427-450. doi: 10.1007/BF02294627

See Also

thetaEst, semTheta, eapEst, eapSem, genPattern, genDichoMatrix , genPolyMatrix ,

nextModule

Examples

## Dichotomous models ##

  # Generation of an item bank under 2PL, made of 7 successive modules that target
 # different average ability levels and with different lengths
 # (the first generated item parameters hold two modules of 8 items each)
 it <- rbind(genDichoMatrix(16, model = "2PL"),
             genDichoMatrix(6, model = "2PL", bPrior = c("norm", -1, 1)),
             genDichoMatrix(6, model = "2PL", bPrior = c("norm", 1, 1)),
             genDichoMatrix(9, model = "2PL", bPrior = c("norm", -2, 1)),
             genDichoMatrix(9, model = "2PL", bPrior = c("norm", 0, 1)),
             genDichoMatrix(9, model = "2PL", bPrior = c("norm", 2, 1)))
 it <- as.matrix(it)

 # Creation of the 'modules' matrix to list item membership in each module
 modules <- matrix(0, 55, 7)
 modules[1:8, 1] <- modules[9:16, 2] <- modules[17:22, 3] <- 1
 modules[23:28, 4] <- modules[29:37, 5] <- modules[38:46, 6] <- 1
 modules[47:55, 7] <- 1

 # Creation of the transition matrix to define a 1-2-3 MST
 trans <- matrix(0, 7, 7)
 trans[1, 3:4] <- trans[2, 3:4] <- trans[3, 5:7] <- trans[4, 5:7] <- 1

 # Creation of the start list: selection by MFI with ability level 0
 start <- list(theta = 0)

 # Creation of the test list: module selection by MFI, ability estimation by WL,
 # stepsize .4 adjustment for constant pattern
 test <- list(method = "WL", moduleSelect = "MFI", constantPatt = "fixed4")

 # Creation of the final list: ability estimation by ML
 final <- list(method = "ML")

 # Random MST generation for true ability level 1 and all ad-interim ability estimates
 res <- randomMST(trueTheta = 1, itemBank = it, modules = modules, transMatrix = trans,
                   start = start, test = test, final = final, allTheta = TRUE) 

 # Module selection by cut-scores for ability estimates
 # Creation of cut-off scores for ability levels: cut score 0 between modules 3 and 4
 # and cut scores -1 and 1 between modules 5, 6 and 7
 # randomesque selection with probability .8
 cut <- rbind(c(3, 4, 0), c(5, 6, -1), c(6, 7, 1))
 test <- list(method = "WL", constantPatt = "fixed4", cutoff = cut, randomesque = 0.8)
 res <- randomMST(trueTheta = 1, itemBank = it, modules = modules, transMatrix = trans,
                   start = start, test = test, final = final, allTheta = TRUE) 

 # Module selection by cut-scores for test scores
 # Creation of cut-off scores for test scores: cut score 4 between modules 3 and 4
 # and cut scores 5 and 9 between modules 5, 6 and 7
 cut.score <- rbind(c(3, 4, 4), c(5, 6, 5), c(6, 7, 9))
 test <- list(method = "score", cutoff = cut.score)
 final <- list(method = "score")
 res <- randomMST(trueTheta = 1, itemBank = it, modules = modules, transMatrix = trans,
                   start = start, test = test, final = final, allTheta = TRUE) 

 # Modification of cut-scores of stage 3 to use only the last module from stage 2 (6 items):
 # cut scores 2 and 4 between modules 5, 6 and 7
 cut.score2 <- rbind(c(3, 4, 4), c(5, 6, 2), c(6, 7, 4))
 test <- list(method = "score", cutoff = cut.score2, score.range = "last")
 final <- list(method = "score")
 res <- randomMST(trueTheta = 1, itemBank = it, modules = modules, transMatrix = trans,
                   start = start, test = test, final = final, allTheta = TRUE) 

 ## Plot options
 plot(trans)
 plot(res)
 plot(res, show.path = FALSE)
 plot(res, border.col = "blue")
 plot(res, arrow.col = "green")