http://worldcat.org/entity/work/id/1909324764

A property of sequential control processes

This memorandum deals with discrete time stochastic control processes having a finite number of states and possible actions. If the process is in state i at time t and one makes decision d sub k, it moves to state j at time t + 1 with probability q sub ij(k). A rule R is a (possibly random) procedure for choosing a decision at each time t given the past up to time t. Each rule R determines a probability P sub R on the set of possible sample paths of the process. Derman (Annals Math. Stat., 35, 1964) has shown that, for criteria involving limiting average probabilities of visits to state-decision pairs, one needs only to consider initial randomizations of nonrandom stationary rules. It is shown that there is a rule which induces the same probability on the process as any initial randomization, hence nothing new is introduced by initial randomization. (Author).

Open All Close All