calculate_validation_statistics {bmstdr}

Calculates the four validation statistics: RMSE, MAE, CRPS and coverage given the observed values and MCMC iterates.

Description

Calculates the four validation statistics: RMSE, MAE, CRPS and coverage given the observed values and MCMC iterates.

Usage

calculate_validation_statistics(yval, yits, level = 95, summarystat = "mean")

Arguments

yval

A vector containing n observed values of the response variable.

yits

A n by N matrix of predictive samples from the n observations contained in yval.

level

The nominal coverage level, defaults to 95%.

summarystat

Summary statistics to use to calculate the validation predictions from the samples. It should be a function like mean or median which can be calculated by R. The default is mean.

Value

A list giving the rmse, mae, crps and coverage.

Examples

set.seed(4)
vrows <- sample(nrow(nysptime), 100)
M1 <- Bsptime(model="lm", formula=y8hrmax~xmaxtemp+xwdsp+xrh, data=nysptime, 
validrows=vrows, scale.transform = "SQRT")
valstats <- calculate_validation_statistics(M1$yobs_preds$y8hrmax, 
yits=t(M1$valpreds))
unlist(valstats)