A Simulation-based Approach to Stochastic Dynamic Programming
Coauthor(s): Nicholas Polson.
In this paper we develop a simulation-based approach to stochastic dynamic programming. To solve the Bellman equation we construct Monte Carlo estimates of Q-values. Our method is scalable to high dimensions and works in both continuous and discrete state and decision spaces whilst avoiding discretization errors that plague traditional methods. We provide a geometric convergence rate. We illustrate our methodology with a dynamic stochastic investment problem.
Source: Applied Stochastic Models in Business and Industry
Polson, Nicholas, and Morten Sorensen. "A Simulation-based Approach to Stochastic Dynamic Programming." Applied Stochastic Models in Business and Industry 27, no. 2 (2011): 151-163.