evaluate_internal {simulator}

Evaluate outputs of methods according to provided metrics.

Description

Given a Metric object or list of Metric objects, this function evaluates an Output object according to these metrics. The computed values of the metrics are saved to file.

Usage

evaluate_internal(
  metrics,
  dir = ".",
  model_name,
  index,
  method_names,
  out_loc = "out"
)

Arguments

metrics

a list of Metric objects or a single Metric object

dir

the directory where Model object was saved (by generate_model)

model_name

the Model object's name attribute

index

the index of a computed Draws object. Can alternately be a vector of such indices.

method_names

the Method objects' name attributes as a character vector.

out_loc

(optional) a length-1 character vector that gives location (relative to model's path) that method outputs are stored.

Details

This function creates objects of class Evals and saves each to file (at dir/model_name/<out_loc>/r<index>_<method_name>_evals.Rdata. Since evaluating metrics is usually (in statistical methodological papers) fast, parallel functionality has not been developed for the evaluation component.