GetAccuracyOverTimePlot {datarobot}

Retrieve Accuracy over Time plot for a model.

Description

Retrieve Accuracy over Time plot for a model.

Usage

GetAccuracyOverTimePlot(
  model,
  backtest = 0,
  source = SourceType$Validation,
  seriesId = NULL,
  forecastDistance = NULL,
  maxBinSize = NULL,
  resolution = NULL,
  startDate = NULL,
  endDate = NULL,
  maxWait = 600
)

Arguments

model

An S3 object of class dataRobotModel like that returned by the function GetModel, or each element of the list returned by the function ListModels.

backtest

integer or character. Optional. Retrieve plots for a specific backtest. Use the backtest index starting from zero. To retrieve plots for holdout, use DataSubset$Holdout.

source

character. Optional. The source of the data for the backtest/holdout. Must be one of SourceType.

seriesId

character. Optional. The name of the series to retrieve for multiseries projects. If not provided an average plot for the first 1000 series will be retrieved.

forecastDistance

integer. Optional. Forecast distance to retrieve the chartdata for. If not specified, the first forecast distance for this project will be used. Only available for time series projects.

maxBinSize

integer. Optional. An int between 1 and 1000, which specifies the maximum number of bins for the retrieval. Default is 500.

resolution

character. Optional. Specifying at which resolution the data should be binned. If not provided an optimal resolution will be used to build chart data with number of bins <= maxBinSize. One of DatetimeTrendPlotsResolutions.

startDate

POSIXct. Optional. The start of the date range to return. If not specified, start date for requested plot will be used.

endDate

POSIXct. Optional. The end of the date range to return. If not specified, end date for requested plot will be used.

maxWait

integer. Optional. The maximum time to wait for a compute job to complete before retrieving the plots. Default is 600. If 0, the plots would be retrieved without attempting the computation.

Value

list with the following components:

• resolution. character: The resolution that is used for binning. One of DatetimeTrendPlotsResolutions.

• startDate. POSIXct: The datetime of the start of the chartdata (inclusive).

• endDate. POSIXct: The datetime of the end of the chartdata (exclusive).

• bins. data.frame: Each row represents a bin in the plot. Dataframe has following columns:

  • startDate. POSIXct: The datetime of the start of the bin (inclusive).

  • endDate. POSIXct: The datetime of the end of the bin (exclusive).

  • actual. numeric: Average actual value of the target in the bin. NA if there are no entries in the bin.

  • predicted. numeric: Average prediction of the model in the bin. NA if there are no entries in the bin.

  • frequency. integer: Indicates number of values averaged in bin.

• statistics. list: Contains statistical properties for the plot.

  • durbinWatson. numeric: The Durbin-Watson statistic for the chart data. Value is between 0 and 4. Durbin-Watson statistic is a test statistic used to detect the presence of autocorrelation at lag 1 in the residuals (prediction errors) from a regression analysis.

• calendarEvents. data.frame: Each row represents a calendar event in the plot. Dataframe has following columns:

  • date. POSIXct: The date of the calendar event.

  • seriesId. character: The series ID for the event. If this event does not specify a series ID, then this will be NA, indicating that the event applies to all series.

  • name. character: The name of the calendar event.

Examples

## Not run: 
projectId <- "59a5af20c80891534e3c2bde"
modelId <- "5996f820af07fc605e81ead4"
model <- GetModel(projectId, modelId)
GetAccuracyOverTimePlot(model)
plot <- GetAccuracyOverTimePlot(model)
png("accuracy_over_time.png", width = 1200, height = 600, units = "px")
par(mar = c(10, 5, 5, 5))
plot(plot$bins$startDate, plot$bins$actual, type = "l", ylab = "Target", xaxt = "n", xlab = "")
lines(plot$bins$startDate, plot$bins$predicted, col = "red")
axis(1, plot$bins$startDate, format(plot$bins$startDate, "%Y-%m-%d"), las = 3)
title(xlab = "Date", mgp = c(7, 1, 0))
legend("topright", legend = c("Actual", "Predicted"), col = c("black", "red"), lty = 1:1)
dev.off()

## End(Not run)