R: Diagnostic Analysis for Emulators

analyze_diagnostic {hmer}

R Documentation

Diagnostic Analysis for Emulators

Description

Produces summary and plots for diagnostics

Usage

analyze_diagnostic(
  in_data,
  output_name,
  targets = NULL,
  plt = interactive(),
  cutoff = 3,
  target_viz = NULL,
  ...
)

Arguments

`in_data`	The data to perform the analysis on
`output_name`	The name of the output emulated
`targets`	If required or desired, the targets for the system outputs
`plt`	Whether or not to plot the analysis
`cutoff`	The implausibility cutoff for diagnostic ‘ce’
`target_viz`	How to show the targets on the diagnostic plots
`...`	Any other parameters to pass to subfunctions

Details

Given diagnostic information (almost certainly provided from get_diagnostic), we can plot the results and highlight the points that are worthy of concern or further consideration. Each diagnostic available has a plot associated with it which can be produced here:

Standardized Error: A histogram of standardized errors. Outliers should be considered, as well as whether very many points have either large or small errors.

Comparison Diagnostics: Error bars around points, corresponding to emulator prediction plus or minus emulator uncertainty. A green line indicates where the emulator and simulator prediction would be in complete agreement: error bars that do not overlap with this line (coloured red) are to be considered. Where targets are provided, the colouration is limited only to points where the simulator prediction would be close to the targets.

Classification Error: A point plot comparing emulator implausibility to simulator implausibility, sectioned into regions horizontally and vertically by cutoff. Points that lie in the lower right quadrant (i.e. emulator would reject; simulator would not) should be considered.

This function takes a data.frame that contains the input points, simulator values and, depending on the diagnostic, a set of summary measures. It returns a data.frame of any points that failed the diagnostic.

We may also superimpose the target bounds on the comparison diagnostics, to get a sense of where it is most important that the emulator and simulator agree. The target_viz argument controls this, and has three options: 'interval' (a horizontal interval); 'solid' (a solid grey box whose dimensions match the target region in both vertical and horizontal extent); and 'hatched' (similar to solid, but a semi-transparent box with hatching inside). Any such vizualisation has extent equal to the target plus/minus 4.5 times the target uncertainty. By default, target_viz = NULL, indicating that no superposition is shown.

Value

A data.frame of failed points

References

Jackson (2018) <http://etheses.dur.ac.uk/12826>