Awi Federgruen

Denumerable undiscounted semi-Markov decision processes with unbounded rewards

Coauthor(s): Paul Schweitzer, H. C. Tijms.


Adobe Acrobat PDF

This paper establishes the existence of a solution to the optimality equations in undiscounted semi-Markov decision models with countable state space, under conditions generalizing the hitherto obtained results. In particular, we merely require the existence of a finite set of states in which every pair of states can reach each other via some stationary policy, instead of the traditional and restrictive assumption that ever stationary policy has a single irreducible set of states. A replacement model and an inventory model illustrate why this extension is essential. Our approach differs fundamentally from classical approaches; we convert the optimality equations into a form suitable for the application of a fixed point theorem.

Source: Mathematics of Operations Research
Exact Citation:
Federgruen, Awi, Paul Schweitzer, and H. C. Tijms. "Denumerable undiscounted semi-Markov decision processes with unbounded rewards." Mathematics of Operations Research 8, no. 2 (May 1983): 298-313.
Volume: 8
Number: 2
Pages: 298-313
Date: 5 1983