vetiver_plot_metrics {vetiver}

Plot model metrics over time for monitoring

Description

These three functions can be used for model monitoring (such as in a monitoring dashboard):

• vetiver_compute_metrics() computes metrics (such as accuracy for a classification model or RMSE for a regression model) at a chosen time aggregation period

• vetiver_pin_metrics() updates an existing pin storing model metrics over time

• vetiver_plot_metrics() creates a plot of metrics over time

Usage

vetiver_plot_metrics(
  df_metrics,
  .index = .index,
  .estimate = .estimate,
  .metric = .metric,
  .n = .n
)

Arguments

df_metrics

A tidy dataframe of metrics over time, such as created by vetiver_compute_metrics().

.index

The variable in df_metrics containing the aggregated dates or date-times (from time_var in data). Defaults to .index.

.estimate

The variable in df_metrics containing the metric estimate. Defaults to .estimate.

.metric

The variable in df_metrics containing the metric type. Defaults to .metric.

.n

The variable in df_metrics containing the number of observations used for estimating the metric.

Value

A ggplot2 object.

See Also

vetiver_compute_metrics(), vetiver_pin_metrics()

Examples

library(dplyr)
library(parsnip)
data(Chicago, package = "modeldata")
Chicago <- Chicago %>% select(ridership, date, all_of(stations))
training_data <- Chicago %>% filter(date < "2009-01-01")
testing_data <- Chicago %>% filter(date >= "2009-01-01", date < "2011-01-01")
monitoring <- Chicago %>% filter(date >= "2011-01-01", date < "2012-12-31")
lm_fit <- linear_reg() %>% fit(ridership ~ ., data = training_data)

library(pins)
b <- board_temp()

## before starting monitoring, initiate the metrics and pin
## (for example, with the testing data):
original_metrics <-
    augment(lm_fit, new_data = testing_data) %>%
    vetiver_compute_metrics(date, "week", ridership, .pred, every = 4L)
pin_write(b, original_metrics, "lm_fit_metrics", type = "arrow")

## to continue monitoring with new data, compute metrics and update pin:
new_metrics <-
    augment(lm_fit, new_data = monitoring) %>%
    vetiver_compute_metrics(date, "week", ridership, .pred, every = 4L)
vetiver_pin_metrics(b, new_metrics, "lm_fit_metrics")

library(ggplot2)
vetiver_plot_metrics(new_metrics) +
    scale_size(range = c(2, 4))