R package pomdp: Partially Observable Markov Decision Processes

CRAN version Rdoc CRAN RStudio mirror downloads CRAN RStudio mirror downloads

Provides the infrastructure to define and analyze the solutions of Partially Observable Markov Decision Processes (POMDP) models. The package includes pomdp-solve (Cassandra, 2015) to solve POMDPs using a variety of algorithms.

The package provides the following algorithms:


Stable CRAN version: install from within R with


Current development version: install from GitHub (needs devtools).



Solving the simple infinite-horizon Tiger problem.

R> library("pomdp")
R> data("Tiger")
R> Tiger
Unsolved POMDP model: Tiger Problem 
    horizon: Inf 
> sol <- solve_POMDP(model = Tiger)
> sol
Solved POMDP model: Tiger Problem 
    solution method: grid 
    horizon: Inf 
    converged: TRUE 
    total expected reward (for start probabilities): 1.933439 
> policy(sol)
  tiger-left tiger-right     action tiger-left tiger-right
1 -98.549921   11.450079  open-left          3           3
2 -10.854299    6.516937     listen          3           1
3   1.933439    1.933439     listen          4           2
4   6.516937  -10.854299     listen          5           3
5  11.450079  -98.549921 open-right          3           3



Development of this package was supported in part by National Institute of Standards and Technology (NIST) under grant number 60NANB17D180.