Comments on the Sensitivity of the Optimal Cost and the Optimal Policy for a Discrete Markov Decision Process.

摘要

The problem of characterizing the effects that uncertainties and/or small changes in the parameters of a model can have an optimal policies is considered. It is shown that changes in the optimal policy are very difficult to detect even for relatively simple models. By showing for a machine replacement problem modeled by a partially observed, finite state Markov decision process, that the infinite horizon, optimal discounted cost function is piecewise linear, we find formulas to compute the optimal cost and the optimal policy, thus providing a means for carrying out sensitivity analyses. Examples are presented to show the usefulness of the results. Keywords: Algorithms; Stochastic control; Dynamic programming. (KR)