Windy_gridworld {pomdp} | R Documentation |
Windy Gridworld MDP
Description
The Windy gridworld MDP example from Chapter 6 of the textbook "Reinforcement Learning: An Introduction."
Format
An object of class MDP.
Details
The gridworld has the following layout:
The grid world is represented as a 7 x 10 matrix of states. In the middle region the next states are shifted upward by wind (the strength in number of squares is given below each column). For example, if the agent is one cell to the right of the goal, then the action left takes the agent to the cell just above the goal.
No discounting is used (i.e., \gamma = 1
).
References
Richard S. Sutton and Andrew G. Barto (2018). Reinforcement Learning: An Introduction Second Edition, MIT Press, Cambridge, MA.
See Also
Other MDP_examples:
Cliff_walking
,
MDP()
,
Maze
Other gridworld:
Cliff_walking
,
Maze
,
gridworld
Examples
data(Windy_gridworld)
Windy_gridworld
gridworld_matrix(Windy_gridworld)
gridworld_matrix(Windy_gridworld, what = "labels")
# The Goal is an absorbing state
which(absorbing_states(Windy_gridworld))
# visualize the transition graph
gridworld_plot_transition_graph(Windy_gridworld,
vertex.size = 10, vertex.label = NA)
# solve using value iteration
sol <- solve_MDP(Windy_gridworld)
sol
policy(sol)
gridworld_plot_policy(sol)