get_dsc_rank {IOHanalyzer}

Get the matrix of rankings using the DSCtool api for a DataSetList

Description

Get the matrix of rankings using the DSCtool api for a DataSetList

Usage

get_dsc_rank(
  dsList,
  targets = NULL,
  which = "by_RT",
  test_type = "AD",
  alpha = 0.05,
  epsilon = 0,
  monte_carlo_iterations = 0,
  na.correction = NULL
)

Arguments

dsList

The DataSetList object

targets

Optional list of target values (Runtime or target value)

which

Whether to use a fixed-target 'by_RT' perspective or fixed-budget 'by_FV'

test_type

Either 'AD' for Anderson-Darling or KS for Kolmogorov-Smirnov tests

alpha

Threshold value for statistical significance

epsilon

Minimum threshold to have practical difference between algorithms (eDSC)

monte_carlo_iterations

How many monte-carlo-simulations to perform (set to 0 to use regular DSC)

na.correction

How to deal with missing values. Only used in fixed-target perspective. Options are: - 'NULL': No correction is done. This will likely result in an error, as the DSCtool does not allow for na values - 'PAR-1' Replace missing values with Budget (budget taken from relevant DataSet) - 'PAR-10' Replace missing values with 10*Budget (budget taken from relevant DataSet) - 'ERT' Replace NA values with the Expected Running Time. If all values are NA, this reverts to nr_runs * budget - 'Remove-na' Removes all NA values

Value

A named list containing a ranked-matrix which has the rankin of each algorithm on each problem, as well as a list of which omnibus tests can be used to further process this data. This can be further analyzed using 'get_dsc_omnibus'

Examples

get_dsc_rank(dsl, na.correction = 'PAR-10')