## Awi Federgruen

*The optimality equation in average cost denumerable state semi-Markov decision problems, recurrency conditions and algorithms*

Coauthor(s): H. C. Tijms.

**Abstract:**

This paper is concerned with the optimality equation for the average costs in a denumerable state semi-Markov decision model. It will be shown that under each of a number of recurrency conditions on the transition probability matrices associated with the stationary policies, the optimality equation has a bounded solution. This solution indeed yields a stationary policy which is optimal for a strong version of the average cost optimality criterion. Besides the existence of a bounded solution to the optimality equation, we will show that both the value-iteration method and the policy-iteration method can be used to determine such a solution. For the latter method we will prove that the average costs and the relative cost functions of the policies generated converge to a solution of the optimality equation.

**Source:** *Journal of Applied Probability*

**Exact Citation:**

Federgruen, Awi, and H. C. Tijms. "The optimality equation in average cost denumerable state semi-Markov decision problems, recurrency conditions and algorithms." *Journal of Applied Probability* 15, no. 2 (June 1978): 356-373.

**Volume:** 15

**Number:** 2

**Pages:** 356-373

**Date:**
6
1978