Sufficient Conditions for the Value Function and Optimal Strategy to be Even and Quasi-Convex

摘要

Sufficient conditions are identified under which the value function and the optimal strategy of a Markov decision process (MDP) are even and quasi-convex in the state. The key idea behind these conditions is the following. First, sufficient conditions for the value function and optimal strategy to be even are identified. Next, it is shown that if the value function and optimal strategy are even, then one can construct a “folded MDP” defined only on the nonnegative values of the state space. Then, the standard sufficient conditions for the value function and optimal strategy to be monotone are “unfolded” to identify sufficient conditions for the value function and the optimal strategy to be quasi-convex. The results are illustrated by using an example of power allocation in remote estimation.