R: Summary of simulated trial results

summary {adaptr}

R Documentation

Summary of simulated trial results

Description

Summarises simulation results from the run_trials() function. Uses extract_results() and check_performance(), which may be used directly to extract key trial results without summarising or to calculate performance metrics (with uncertainty measures if desired) and return them in a tidy data.frame.

Usage

## S3 method for class 'trial_results'
summary(
  object,
  select_strategy = "control if available",
  select_last_arm = FALSE,
  select_preferences = NULL,
  te_comp = NULL,
  raw_ests = FALSE,
  final_ests = NULL,
  restrict = NULL,
  cores = NULL,
  ...
)

Arguments

`object`	`trial_results` object, output from the `run_trials()` function.
`select_strategy`	single character string. If a trial was not stopped due to superiority (or had only 1 arm remaining, if `select_last_arm` is set to `TRUE` in trial designs with a common `control` arm; see below), this parameter specifies which arm will be considered selected when calculating trial design performance metrics, as described below; this corresponds to the consequence of an inconclusive trial, i.e., which arm would then be used in practice. The following options are available and must be written exactly as below (case sensitive, cannot be abbreviated): `"control if available"` (default): selects the first `control` arm for trials with a common `control` arm if this arm is active at end-of-trial, otherwise no arm will be selected. For trial designs without a common `control`, no arm will be selected. `"none"`: selects no arm in trials not ending with superiority. `"control"`: similar to `"control if available"`, but will throw an error if used for trial designs without a common `control` arm. `"final control"`: selects the final `control` arm regardless of whether the trial was stopped for practical equivalence, futility, or at the maximum sample size; this strategy can only be specified for trial designs with a common `control` arm. `"control or best"`: selects the first `control` arm if still active at end-of-trial, otherwise selects the best remaining arm (defined as the remaining arm with the highest probability of being the best in the last adaptive analysis conducted). Only works for trial designs with a common `control` arm. `"best"`: selects the best remaining arm (as described under `"control or best"`). `"list or best"`: selects the first remaining arm from a specified list (specified using `select_preferences`, technically a character vector). If none of these arms are are active at end-of-trial, the best remaining arm will be selected (as described above). `"list"`: as specified above, but if no arms on the provided list remain active at end-of-trial, no arm is selected.
`select_last_arm`	single logical, defaults to `FALSE`. If `TRUE`, the only remaining active arm (the last `control`) will be selected in trials with a common `control` arm ending with `equivalence` or `futility`, before considering the options specified in `select_strategy`. Must be `FALSE` for trial designs without a common `control` arm.
`select_preferences`	character vector specifying a number of arms used for selection if one of the `"list or best"` or `"list"` options are specified for `select_strategy`. Can only contain valid `arms` available in the trial.
`te_comp`	character string, treatment-effect comparator. Can be either `NULL` (the default) in which case the first `control` arm is used for trial designs with a common control arm, or a string naming a single trial `arm`. Will be used when calculating `err_te` and `sq_err_te` (the error and the squared error of the treatment effect comparing the selected arm to the comparator arm, as described below).
`raw_ests`	single logical. If `FALSE` (default), the posterior estimates (`post_ests` or `post_ests_all`, see `setup_trial()` and `run_trial()`) will be used to calculate `err` and `sq_err` (the error and the squared error of the estimated compared to the specified effect in the selected arm) and `err_te` and `sq_err_te` (the error and the squared error of the treatment effect comparing the selected arm to the comparator arm, as described for `te_comp` and below). If `TRUE`, the raw estimates (`raw_ests` or `raw_ests_all`, see `setup_trial()` and `run_trial()`) will be used instead of the posterior estimates.
`final_ests`	single logical. If `TRUE` (recommended) the final estimates calculated using outcome data from all patients randomised when trials are stopped are used (`post_ests_all` or `raw_ests_all`, see `setup_trial()` and `run_trial()`); if `FALSE`, the estimates calculated for each arm when an arm is stopped (or at the last adaptive analysis if not before) using data from patients having reach followed up at this time point and not all patients randomised are used (`post_ests` or `raw_ests`, see `setup_trial()` and `run_trial()`). If `NULL` (the default), this argument will be set to `FALSE` if outcome data are available immediate after randomisation for all patients (for backwards compatibility, as final posterior estimates may vary slightly in this situation, even if using the same data); otherwise it will be said to `TRUE`. See `setup_trial()` for more details on how these estimates are calculated.
`restrict`	single character string or `NULL`. If `NULL` (default), results are summarised for all simulations; if `"superior"`, results are summarised for simulations ending with superiority only; if `"selected"`, results are summarised for simulations ending with a selected arm only (according to the specified arm selection strategy for simulations not ending with superiority). Some summary measures (e.g., `prob_conclusive`) have substantially different interpretations if restricted, but are calculated nonetheless.
`cores`	`NULL` or single integer. If `NULL`, a default value set by `setup_cluster()` will be used to control whether extractions of simulation results are done in parallel on a default cluster or sequentially in the main process; if a value has not been specified by `setup_cluster()`, `cores` will then be set to the value stored in the global `"mc.cores"` option (if previously set by `⁠options(mc.cores = <number of cores>⁠`), and `1` if that option has not been specified. If `cores = 1`, computations will be run sequentially in the primary process, and if `cores > 1`, a new parallel cluster will be setup using the `parallel` library and removed once the function completes. See `setup_cluster()` for details.
`...`	additional arguments, not used.

Value

A "trial_results_summary" object containing the following values:

n_rep: the number of simulations.
n_summarised: as described in check_performance().
highest_is_best: as specified in setup_trial().
elapsed_time: the total simulation time.
size_mean, size_sd, size_median, size_p25, size_p75, size_p0, size_p100, sum_ys_mean, sum_ys_sd, sum_ys_median, sum_ys_p25, sum_ys_p75, sum_ys_p0, sum_ys_p100, ratio_ys_mean, ratio_ys_sd, ratio_ys_median, ratio_ys_p25, ratio_ys_p75, ratio_ys_p0, ratio_ys_p100, prob_conclusive, prob_superior, prob_equivalence, prob_futility, prob_max, ⁠prob_select_*⁠ (with * being either "⁠arm_<name>⁠ for all arm names or none), rmse, rmse_te, mae, mae_te, and idp: performance metrics as described in check_performance(). Note that all sum_ys_ and ratio_ys_ measures use outcome data from all randomised patients, regardless of whether they had outcome data available at the last analysis or not, as described in extract_results().
select_strategy, select_last_arm, select_preferences, te_comp, raw_ests, final_ests, restrict: as specified above.
control: the control arm specified by setup_trial(), setup_trial_binom() or setup_trial_norm(); NULL if no control.
equivalence_assessed, futility_assessed: single logicals, specifies whether the trial design specification includes assessments of equivalence and/or futility.
base_seed: as specified in run_trials().
cri_width, n_draws, robust, description, add_info: as specified in setup_trial(), setup_trial_binom() or setup_trial_norm().

Examples

# Setup a trial specification
binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
                                 control = "A",
                                 true_ys = c(0.20, 0.18, 0.22, 0.24),
                                 data_looks = 1:20 * 100)

# Run 10 simulations with a specified random base seed
res <- run_trials(binom_trial, n_rep = 10, base_seed = 12345)

# Summarise simulations - select the control arm if available in trials not
# ending with a superiority decision
res_sum <- summary(res, select_strategy = "control")

# Print summary
print(res_sum, digits = 1)