await {civis}

Call a function repeatedly until a status is reached.

Description

await repeatedly calls a Civis API endpoint such as scripts_get_sql_runs that monitors the status of a script, job, import, or model. It blocks until the function returns a result with a successful or error status. If the script, job, import or model results in an error state, await throws an error with useful debugging information.

await_all is a vectorized version of await. It repeatedly calls a Civis API endpoint for all values of a vector, e.g. a vector of script, job, import, run, or model ids. It blocks until all calls have returned a result with a given status, and silently captures jobs that return errors.

Usage

await(
  f,
  ...,
  .status_key = "state",
  .success_states = c("succeeded", "success"),
  .error_states = c("failed", "cancelled"),
  .timeout = NULL,
  .interval = getOption("civis.default_polling_interval"),
  .verbose = FALSE
)

await_all(
  f,
  .x,
  .y = NULL,
  ...,
  .status_key = "state",
  .success_states = c("succeeded", "success"),
  .error_states = c("failed", "cancelled"),
  .timeout = NULL,
  .interval = NULL,
  .verbose = FALSE
)

Arguments

f

function to be called repeatedly until a status is reached.

...

arguments to f

.status_key

The name of the element of the list returned by f containing the status. For most Civis API endpoints, this is the default, "state".

.success_states

list of states indicating remote work has completed successfully. For most Civis API endpoints, this set of states is the default, c("succeeded", "success").

.error_states

list of states indicating remote work is in an error state. For most Civis API endpoints, this set of states is the default, c("failed", "cancelled").

.timeout

Number of seconds after which to timeout.

.interval

The interval for retries (in seconds). If NULL (default), use exponentially increasing intervals with jitter (see 'Details')

.verbose

Print the status of f at a given retry with the retry time (default FALSE)

.x

a vector of values to be passed to f

.y

a vector of values to be passed to f (default NULL)

Details

await and await_all can wrap Civis API endpoints in generated_client.R. The default values for .status_key, .success_states, and .error_states are suitable for most endpoints. The final status of f can be obtained using get_status.

If an error state is reached, await throws a civis_await_error. await_all silently captures and returns a civis_await_error for any job reaching an error state as an element in the list of results.

If .timeout is specified and the job fails to reach a success state within the time limit, await throws a civis_timeout_error. Likewise, await_all throws a civis_timeout_error if all jobs fail to reach a success state within the time limit.

These errors can be caught using try or tryCatch. Useful debugging information can be returned using get_error and fetch_logs.

The set of possible states for jobs on Civis platform are: "succeeded", "success", "failed", "queued", "running", and "cancelled".

Unless .interval is specified, retries are attempted with exponentially increasing intervals using .25 * (1.2^i)) + runif(1, 0, .2), where i is the index of the current retry. Approximate intervals for a given number of retries are as follows:

• 1-5: .5s

• 6-10: 1-5s

• 11-19: 5-10s

• 20-29: 10s - 1m

The polling interval can be set to a fixed value globally with options("civis.default_polling_interval" = INTERVAL_IN_SECONDS).

Functions

• await_all(): Call a function repeatedly for all values of a vector until all have reached a completed status

See Also

get_status, get_error, fetch_logs

Examples

## Not run: 

   # Executing a query
   q_id <- queries_post(db_id, query, n_rows, cred_id)[["id"]]
   r <- await(queries_get, id = q_id)
   get_status(r)

   r <- tryCatch(await(queries_get, id = q_id), error = function(e) e)
   get_error(r)

   r <- try(await(queries_get, id = q_id))
   get_error(r)

   jobs <- c(1234, 5678)
   runs <- c(1234, 5678)
   rs <- await_all(scripts_get_r_runs, .x = jobs, .y = runs)

## End(Not run)