This work studies the problem of adaptive online optimization of Markov reward processes. The problem at hand is the following: given a Markov chain whose transition probability matrix and its expected cost per stage are functions of a (1) a set of tunable parameters, and (2) a set of unknown but fixed parameters, find the set of (tunable) parameters that maximizes the average reward per stage observed. This work introduces techniques that improve the performance of existing simulation-based methods, and that are robust to uncertainty of the system parameters. We show the almost sure convergence of the algorithms to locally optimal values, including the adaptive case, while the tracking ability of the adaptive algorithm is illustrated numerically.; The methodological work in online methods is applied to a significant optimization problem, namely the problem of setting prices for services in a multiclass loss networks. Such networks consists of a set of resources shared by multiple classes of users characterized by their usage patterns. The network sets the price per-call/per-class and it is assumed that users are sensitive to prices, in the sense that prices affect the arrival process. The algorithms developed here are applied to the solution to this problem. The tracking ability of the algorithms is illustrated by scenarios where the service time parameters change smoothly, or infrequently, over time.
展开▼
