R: Estimate Truncation of Observed Data

estimate_truncation {EpiNow2}

R Documentation

Estimate Truncation of Observed Data

Description

Estimates a truncation distribution from multiple snapshots of the same data source over time. This distribution can then be used passed to the truncation argument in regional_epinow(), epinow(), and estimate_infections() to adjust for truncated data and propagate the uncertainty associated with data truncation into the estimates.

See here for an example of using this approach on Covid-19 data in England. The functionality offered by this function is now available in a more principled manner in the epinowcast R package.

The model of truncation is as follows:

The truncation distribution is assumed to be discretised log normal wit a mean and standard deviation that is informed by the data.
The data set with the latest observations is adjusted for truncation using the truncation distribution.
Earlier data sets are recreated by applying the truncation distribution to the adjusted latest observations in the time period of the earlier data set. These data sets are then compared to the earlier observations assuming a negative binomial observation model with an additive noise term to deal with zero observations.

This model is then fit using stan with standard normal, or half normal, prior for the mean, standard deviation, 1 over the square root of the overdispersion and additive noise term.

This approach assumes that:

Current truncation is related to past truncation.
Truncation is a multiplicative scaling of underlying reported cases.
Truncation is log normally distributed.

Usage

estimate_truncation(
  data,
  max_truncation,
  trunc_max = 10,
  trunc_dist = "lognormal",
  truncation = trunc_opts(LogNormal(meanlog = Normal(0, 1), sdlog = Normal(1, 1), max =
    10)),
  model = NULL,
  stan = stan_opts(),
  CrIs = c(0.2, 0.5, 0.9),
  filter_leading_zeros = FALSE,
  zero_threshold = Inf,
  weigh_delay_priors = FALSE,
  verbose = TRUE,
  ...,
  obs
)

Arguments

`data`	A list of `⁠<data.frame>⁠`s each containing a date variable and a confirm (numeric) variable. Each data set should be a snapshot of the reported data over time. All data sets must contain a complete vector of dates.
`max_truncation`	Deprecated; use `truncation` instead.
`trunc_max`	Deprecated; use `truncation` instead.
`trunc_dist`	Deprecated; use `truncation` instead.
`truncation`	A call to `trunc_opts()` defining the truncation of the observed data. Defaults to `trunc_opts()`, i.e. no truncation. See the `estimate_truncation()` help file for an approach to estimating this from data where the `dist` list element returned by `estimate_truncation()` is used as the `truncation` argument here, thereby propagating the uncertainty in the estimate.
`model`	A compiled stan model to override the default model. May be useful for package developers or those developing extensions.
`stan`	A list of stan options as generated by `stan_opts()`. Defaults to `stan_opts()`. Can be used to override `data`, `init`, and `verbose` settings if desired.
`CrIs`	Numeric vector of credible intervals to calculate.
`filter_leading_zeros`	Logical, defaults to TRUE. Should zeros at the start of the time series be filtered out.
`zero_threshold`	Numeric defaults to Inf. Indicates if detected zero cases are meaningful by using a threshold number of cases based on the 7-day average. If the average is above this threshold then the zero is replaced using `fill`.
`weigh_delay_priors`	Deprecated; use the `weight_prior` option in `trunc_opts()` instead.
`verbose`	Logical, should model fitting progress be returned.
`...`	Additional parameters to pass to `rstan::sampling()`.
`obs`	Deprecated; use `data` instead.

Value

A list containing: the summary parameters of the truncation distribution (dist), which could be passed to the truncation argument of epinow(), regional_epinow(), and estimate_infections(), the estimated CMF of the truncation distribution (cmf, can be used to adjusted new data), a ⁠<data.frame>⁠ containing the observed truncated data, latest observed data and the adjusted for truncation observations (obs), a ⁠<data.frame>⁠ containing the last observed data (last_obs, useful for plotting and validation), the data used for fitting (data) and the fit object (fit).

Examples


# set number of cores to use
old_opts <- options()
options(mc.cores = ifelse(interactive(), 4, 1))

# fit model to example data
# See [example_truncated] for more details
est <- estimate_truncation(example_truncated,
  verbose = interactive(),
  chains = 2, iter = 2000
)

# summary of the distribution
est$dist
# summary of the estimated truncation cmf (can be applied to new data)
print(est$cmf)
# observations linked to truncation adjusted estimates
print(est$obs)
# validation plot of observations vs estimates
plot(est)

# Pass the truncation distribution to `epinow()`.
# Note, we're using the last snapshot as the observed data as it contains
# all the previous snapshots. Also, we're using the default options for
# illustrative purposes only.
out <- epinow(
  example_truncated[[5]],
  truncation = trunc_opts(est$dist)
)
plot(out)
options(old_opts)

[Package EpiNow2 version 1.5.2 Index]