Morten Sorensen

A Simulation-based Approach to Stochastic Dynamic Programming

Coauthor(s): Nicholas Polson.

Abstract:
In this paper we develop a simulation-based approach to stochastic dynamic programming. To solve the Bellman equation we construct Monte Carlo estimates of Q-values. Our method is scalable to high dimensions and works in both continuous and discrete state and decision spaces whilst avoiding discretization errors that plague traditional methods. We provide a geometric convergence rate. We illustrate our methodology with a dynamic stochastic investment problem.

Source: Applied Stochastic Models in Business and Industry
Exact Citation:
Polson, Nicholas, and Morten Sorensen. "A Simulation-based Approach to Stochastic Dynamic Programming." Applied Stochastic Models in Business and Industry 27, no. 2 (2011): 151-163.
Volume: 27
Number: 2
Pages: 151-163
Date: 2011