knn.forecast.randomsearch.tuning {knnwtsim}

Tune knn.forecast() Hyperparameters with Random Search

Description

A simplistic automated hyperparameter tuning function which randomly generates a grid of hyperparameter sets used to build corresponding S_w similarity matrices which are used in knn.forecast() test against the last test.h points of y.in after any val.holdout.len points are removed from the end of y.in. The best performing set of parameters based on MAPE over over the forecast horizon of test.h points are returned as part of a list alongside the 'optimum' weighted similarity matrix Sw.opt, the Grid of tested sets, and the MAPE results. MAPE is the average of absolute percent errors for each point calculated as: abs((test.actuals - test.forecast.i) / test.actuals) * 100. Where test.forecast.i and test.actuals are both numeric vectors.

Usage

knn.forecast.randomsearch.tuning(
  grid.len = 100,
  St.in,
  Sp.in,
  Sx.in,
  y.in,
  test.h = 1,
  max.k = NULL,
  val.holdout.len = 0,
  min.k = 1
)

Arguments

grid.len

integer value representing the number of hyperparameter sets to generate and test, must be >= 1.

St.in

numeric and symmetric matrix of similarities, can be generated with StMatrixCalc().

Sp.in

numeric and symmetric matrix of similarities, can be generated with SpMatrixCalc().

Sx.in

numeric and symmetric matrix of similarities, can be generated with SxMatrixCalc().

y.in

numeric vector of the response series to be forecast.

test.h

integer value representing the number of points in the test forecast horizon, must be >= 1.

max.k

integer value representing the maximum value of k, knn.forecast() should use, will be set to min(floor((length(y.in)) * .4), length(y.in) - val.holdout.len - test.h) if NULL or NA is passed. Note this NA behavior differs from knnwtsim version 0.1.0. If a numeric value is passed it must be >= 1.

val.holdout.len

integer value representing the number of observations at the end of the series to be removed in testing forecast if desired to leave a validation set after tuning, must be >= 0.

min.k

integer value representing the minimum value of k, knn.forecast() should use, must be >= 1.

Value

list of the following components:

weight.opt

numeric vector of the 3 weights to generate Sw.opt in alpha, beta, gamma order which achieved the best performance in terms of MAPE.

k.opt

integer value of neighbors used in knn.forecast() which achieved the best performance in terms of MAPE.

Sw.opt

numeric matrix of similarities calculated using S_w, with the best performing set of hyperparameters.

Test.MAPE

numeric value of the MAPE result for the optimum hyperparamter set achieved on the test points.

MAPE.all

numeric vector of MAPE results, each observation corresponds to the row in Grid of the same index.

Grid

dataframe of all hyperparameter sets tested in the tuning.

See Also

• Trupiano (2021) arXiv:2112.06266 for information on the formulation of S_w.

• StMatrixCalc() for information on the calculation of S_t.

• SpMatrixCalc() for information on the calculation of S_p.

• SxMatrixCalc() for information on the calculation of S_x.

• knn.forecast() for the function called to perform knn regression.

Examples

data("simulation_master_list")
series.index <- 15
ex.series <- simulation_master_list[[series.index]]$series.lin.coef.chng.x

df <- data.frame(ex.series)
# Generate vector of time orders
df$t <- c(1:nrow(df))
# Generate vector of periods
nperiods <- simulation_master_list[[series.index]]$seasonal.periods
df$p <- rep(1:nperiods, length.out = nrow(df))
# Pull corresponding exogenous predictor(s)
X <- as.matrix(simulation_master_list[[series.index]]$x.chng)

St.ex <- StMatrixCalc(df$t)
Sp.ex <- SpMatrixCalc(df$p, nPeriods = nperiods)
Sx.ex <- SxMatrixCalc(X)

tuning.test <- knn.forecast.randomsearch.tuning(
  grid.len = 10,
  y.in = ex.series,
  St.in = St.ex,
  Sp.in = Sp.ex,
  Sx.in = Sx.ex,
  test.h = 3,
  max.k = 10,
  val.holdout.len = 3
)