The optimality equation in average cost denumerable state semi-Markov decision problems, recurrency conditions and algorithms
Coauthor(s): H. C. Tijms.
This paper is concerned with the optimality equation for the average costs in a denumerable state semi-Markov decision model. It will be shown that under each of a number of recurrency conditions on the transition probability matrices associated with the stationary policies, the optimality equation has a bounded solution. This solution indeed yields a stationary policy which is optimal for a strong version of the average cost optimality criterion. Besides the existence of a bounded solution to the optimality equation, we will show that both the value-iteration method and the policy-iteration method can be used to determine such a solution. For the latter method we will prove that the average costs and the relative cost functions of the policies generated converge to a solution of the optimality equation.
Source: Journal of Applied Probability
Federgruen, Awi, and H. C. Tijms. "The optimality equation in average cost denumerable state semi-Markov decision problems, recurrency conditions and algorithms." Journal of Applied Probability 15, no. 2 (June 1978): 356-373.