nemenyi {tsutils}

Nonparametric multiple comparisons (Nemenyi test)

Description

Perform nonparametric multiple comparisons, across columns, using the Friedman and the post-hoc Nemenyi tests.

Usage

nemenyi(
  data,
  conf.level = 0.95,
  sort = c(TRUE, FALSE),
  plottype = c("vline", "none", "mcb", "vmcb", "line", "matrix"),
  select = NULL,
  labels = NULL,
  ...
)

Arguments

data

an array that includes values to be compared for several treatments (in columns) for several observations (rows), of size n x k. For example, if these are forecast errors, different methods should be in columns and errors for different time series or forecast origins in rows.

conf.level

the confidence level used for the comparison. Default is 0.95.

sort

if TRUE, then function sorts the outputted values of mean ranks. If plots are request, this is forced to TRUE.

plottype

type of plot to produce:

• "none": no plot.

• "mcb": Multiple Comparison with the Best style plot.

• "vmcb": vertical MCB plot.

• "line": summarised line plot.

• "vline": vertical line plot.

• "matrix": complete matrix visualisation.

select

highlight selected treatment (column). Number 1 to k. Use NULL for no highlighting.

labels

optional labels for models. If NULL column names of data will be used.

...

additional arguments passed to the plot function.

Value

Return object of class nemenyi and contains:

• means: mean rank of each treatment.

• intervals: intervals within there is no evidence of significance difference according to the Nemenyi test at requested confidence level.

• fpavl: Friedman test p-value.

• fH: Friedman test hypothesis outcome.

• cd: Nemenyi critical distance. Output intervals is calculate as means +/- cd.

• conf.level: confidence level used for testing.

• k: number of treatments (columns).

• n: number of observations (rows).

Author(s)

Nikolaos Kourentzes, [e-mail],

Ivan Svetunkov, [e-mail].

References

• The tests are deailed by Hollander, M., Wolfe, D.A. and Chicken, E. (2014) Nonparametric Statistical Methods. 3rd Edition, John Wiley & Sons, Inc., New York.

• The line plot is introduced here and a first example of its use, along with a short description is provided by Kourentzes, N. (2013). Intermittent demand forecasts with neural networks. International Journal of Production Economics, 143(1), 198-206.

• The matrix plot is introduced by Kourentzes, N., & Athanasopoulos, G. (2018). Cross-temporal coherent forecasts for Australian tourism (No. 24/18). Monash University, Department of Econometrics and Business Statistics.

• The MCB plot is described by Koning, A. J., Franses, P. H., Hibon, M., & Stekler, H. O. (2005). The M3 competition: Statistical tests of the results. International Journal of Forecasting, 21(3), 397-409.

Examples

x <- matrix( rnorm(50*4,mean=0,sd=1), 50, 4)
x[,2] <- x[,2]+1
x[,3] <- x[,3]+0.7
x[,4] <- x[,4]+0.5
colnames(x) <- c("Method A","Method B","Method C - long name","Method D")
nemenyi(x,conf.level=0.95,plottype="vline")