score_index {FoReco}

Measuring accuracy in a rolling forecast experiment

Description

Function to calculate the accuracy indices of the reconciled point forecasts of a cross-temporal (not only, see examples) system (more in Average relative accuracy indices). (Experimental version)

Usage

score_index(recf, base, test, m, nb, nl, type = "mse", compact = TRUE)

Arguments

recf

list of q (forecast origins) reconciled forecasts' matrices (\([n \times h(k^\ast + m)]\) in the cross-temporal case, \([h \times n]\) in the cross-sectional case, and vectors of length \([h(k^\ast \times m)]\) in the temporal framework).

base

list of q (forecast origins) base forecasts' matrices (\([n \times h(k^\ast + m)]\) in the cross-temporal case, \([h \times n]\) in the cross-sectional case, and vectors of length \([h(k^\ast \times m)]\) in the temporal framework).

test

list of q (forecast origins) test observations' matrices (\([n \times h(k^\ast + m)]\) in the cross-temporal case, \([h \times n]\) in the cross-sectional case, and vectors of length \([h(k^\ast \times m)]\) in the temporal framework).

m

Highest available sampling frequency per seasonal cycle (max. order of temporal aggregation, \(m\)), or a subset of \(p\) factors of \(m\).

nb

number of bottom time series in the cross-sectional framework.

nl

(\(L \times 1\)) vector containing the number of time series in each cross-sectional level of the hierarchy (nl[1] = 1).

type

type of accuracy measure ("mse" Mean Square Error, "rmse" Root Mean Square Error or "mae" Mean Absolute Error).

compact

if TRUE returns only the summary matrix.

Value

It returns a summary table called Avg_mat (if compact option is TRUE, default), otherwise it returns a list of six tables (more in Average relative accuracy indices).

References

Di Fonzo, T., and Girolimetto, D. (2023), Cross-temporal forecast reconciliation: Optimal combination method and heuristic alternatives, International Journal of Forecasting, 39(1), 39-57.

See Also

Other utilities: Cmatrix(), FoReco2ts(), agg_ts(), arrange_hres(), commat(), ctf_tools(), hts_tools(), lcmat(), oct_bounds(), residuals_matrix(), shrink_estim(), thf_tools()

Examples

data(FoReco_data)

# Cross-temporal framework
oct_recf <- octrec(FoReco_data$base, m = 12, C = FoReco_data$C,
                   comb = "bdshr", res = FoReco_data$res)$recf
oct_score <- score_index(recf = oct_recf,
                         base = FoReco_data$base,
                         test = FoReco_data$test, m = 12, nb = 5)

# Cross-sectional framework
# monthly base forecasts
mbase <- FoReco2matrix(FoReco_data$base, m = 12)$k1
# monthly test set
mtest <- FoReco2matrix(FoReco_data$test, m = 12)$k1
# monthly residuals
mres <- FoReco2matrix(FoReco_data$res, m = 12)$k1
# monthly reconciled forecasts
mrecf <- htsrec(mbase, C = FoReco_data$C, comb = "shr", res = mres)$recf
# score
hts_score <- score_index(recf = mrecf, base = mbase, test = mtest, nb = 5)

# Temporal framework
# top ts base forecasts ([lowest_freq' ...  highest_freq']')
topbase <- FoReco_data$base[1, ]
# top ts residuals ([lowest_freq' ...  highest_freq']')
topres <- FoReco_data$res[1, ]
# top ts test ([lowest_freq' ...  highest_freq']')
toptest <- FoReco_data$test[1, ]
# top ts recf ([lowest_freq' ...  highest_freq']')
toprecf <- thfrec(topbase, m = 12, comb = "acov", res = topres)$recf
# score
thf_score <- score_index(recf = toprecf, base = topbase, test = toptest, m = 12)