R: Create Policy Data Object

policy_data {polle}

R Documentation

Create Policy Data Object

Description

policy_data() creates a policy data object which is used as input to policy_eval() and policy_learn() for policy evaluation and data adaptive policy learning.

Usage

policy_data(
  data,
  baseline_data,
  type = "wide",
  action,
  covariates,
  utility,
  baseline = NULL,
  deterministic_rewards = NULL,
  id = NULL,
  stage = NULL,
  event = NULL,
  action_set = NULL,
  verbose = FALSE
)

## S3 method for class 'policy_data'
print(x, digits = 2, ...)

## S3 method for class 'policy_data'
summary(object, probs = seq(0, 1, 0.25), ...)

Arguments

`data`	data.frame or data.table; see Examples.
`baseline_data`	data.frame or data.table; see Examples.
`type`	Character string. If "wide", `data` is considered to be on wide format. If "long", `data` is considered to be on long format; see Examples.
`action`	Action variable name(s). Character vector or character string. A vector is valid for wide data. The length of the vector determines the number of stages (K). A string is valid for single stage wide data or long data.
`covariates`	Stage specific covariate name(s). Character vector or named list of character vectors. A vector is valid for single stage wide data or long data. A named list is valid for multiple stages wide data. Each element must be a character vector with length K. Each vector can contain NA elements, if a covariate is not available for the given stage(s).
`utility`	Utility/Reward variable name(s). Character string or vector. A string is valid for long data and wide data with a single final utility. A vector is valid for wide data with incremental rewards. Must have length K+1; see Examples.
`baseline`	Baseline covariate name(s). Character vector.
`deterministic_rewards`	Deterministic reward variable name(s). Named list of character vectors of length K. The name of each element must be on the form "U_Aa" where "a" corresponds to an action in the action set.
`id`	ID variable name. Character string.
`stage`	Stage number variable name.
`event`	Event indicator name.
`action_set`	Character string. Action set across all stages.
`verbose`	Logical. If TRUE, formatting comments are printed to the console.
`x`	Object to be printed.
`digits`	Minimum number of digits to be printed.
`...`	Additional arguments passed to print.
`object`	Object of class policy_data
`probs`	numeric vector (probabilities)

Details

Each observation has the sequential form

O= {B, U_1, X_1, A_1, ..., U_K, X_K, A_K, U_{K+1}},

for a possibly stochastic number of stages K.

B is a vector of baseline covariates.
U_k is the reward at stage k (not influenced by the action A_k).
X_k is a vector of state covariates summarizing the state at stage k.
A_k is the categorical action at stage k.

The utility is given by the sum of the rewards, i.e., U = \sum_{k = 1}^{K+1} U_k.

Value

policy_data() returns an object of class "policy_data". The object is a list containing the following elements:

`stage_data`	data.table containing the id, stage number, event indicator, action (`A_k`), state covariates (`X_k`), reward (`U_k`), and the deterministic rewards.
`baseline_data`	data.table containing the id and baseline covariates (`B`).
`colnames`	List containing the state covariate names, baseline covariate names, and the deterministic reward variable names.
`action_set`	Sorted character vector describing the action set, i.e., the possible actions at all stages.
`stage_action_sets`	List of sorted character vectors describing the observed actions at each stage.
`dim`	List containing the number of observations (n) and the number of stages (K).

S3 generics

The following S3 generic functions are available for an object of class policy_data:

partial(): Trim the maximum number of stages in a policy_data object.
subset_id(): Subset a a policy_data object on ID.
get_history(): Summarize the history and action at a given stage.
get_history_names(): Get history variable names.
get_actions(): Get the action at every stage.
get_utility(): Get the utility.
plot(): Plot method.

Examples

library("polle")
### Single stage: Wide data
d1 <- sim_single_stage(n = 5e2, seed=1)
head(d1, 5)
# constructing policy_data object:
pd1 <- policy_data(d1,
                   action="A",
                   covariates=c("Z", "B", "L"),
                   utility="U")
pd1
# associated S3 methods:
methods(class = "policy_data")
head(get_actions(pd1), 5)
head(get_utility(pd1), 5)
head(get_history(pd1)$H, 5)

### Two stage: Wide data
d2 <- sim_two_stage(5e2, seed=1)
head(d2, 5)
# constructing policy_data object:
pd2 <- policy_data(d2,
                  action = c("A_1", "A_2"),
                  baseline = c("B"),
                  covariates = list(L = c("L_1", "L_2"),
                                    C = c("C_1", "C_2")),
                  utility = c("U_1", "U_2", "U_3"))
pd2
head(get_history(pd2, stage = 2)$H, 5) # state/Markov type history and action, (H_k,A_k).
head(get_history(pd2, stage = 2, full_history = TRUE)$H, 5) # Full history and action, (H_k,A_k).

### Multiple stages: Long data
d3 <- sim_multi_stage(5e2, seed = 1)
head(d3$stage_data, 10)
# constructing policy_data object:
pd3 <- policy_data(data = d3$stage_data,
                   baseline_data = d3$baseline_data,
                   type = "long",
                   id = "id",
                   stage = "stage",
                   event = "event",
                   action = "A",
                   utility = "U")
pd3
head(get_history(pd3, stage = 3)$H, 5) # state/Markov type history and action, (H_k,A_k).
head(get_history(pd3, stage = 2, full_history = TRUE)$H, 5) # Full history and action, (H_k,A_k).

[Package polle version 1.4 Index]