Denumerable undiscounted semi-Markov decision processes with unbounded rewards
Coauthor(s): Paul Schweitzer, H. C. Tijms.
Adobe Acrobat PDF
This paper establishes the existence of a solution to the optimality equations in undiscounted semi-Markov decision models with countable state space, under conditions generalizing the hitherto obtained results. In particular, we merely require the existence of a finite set of states in which every pair of states can reach each other via some stationary policy, instead of the traditional and restrictive assumption that ever stationary policy has a single irreducible set of states. A replacement model and an inventory model illustrate why this extension is essential. Our approach differs fundamentally from classical approaches; we convert the optimality equations into a form suitable for the application of a fixed point theorem.
Source: Mathematics of Operations Research
Federgruen, Awi, Paul Schweitzer, and H. C. Tijms. "Denumerable undiscounted semi-Markov decision processes with unbounded rewards." Mathematics of Operations Research 8, no. 2 (May 1983): 298-313.