### Abstract

We address the adaptive stochastic control problem for a discrete time system described by controlled Markov chain with finite number of states. The mirror descent randomized control algorithm on the class of controlled homogeneous finite Markov chains with unknown mean losses has been proposed and studied. Here we develop the approach represented in Nazin and Miller (2011). The main assumptions are the following: processes are independent and stationary, nonnegative random losses are almost surely bounded by a given constant, and the connectivity assumption for the controlled Markov chain holds. The uncertainty is that the mean loss matrix is unknown. The novelty of the approach is in extension of the class of controlled homogeneous finite Markov chains to the chains with connectivity assumption. The main result consists in demonstration of the asymptotical upper bound (that is asymptotic by time) and in determining the explicit constant which is weakly depending on the logarithm of the number of states.

Original language | English |
---|---|

Title of host publication | Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC) |

Editors | Marios Polycarpou |

Place of Publication | USA |

Publisher | IEEE, Institute of Electrical and Electronics Engineers |

Pages | 1779 - 1783 |

Number of pages | 5 |

DOIs | |

Publication status | Published - 2011 |

Event | IEEE Conference of Decision and Control (CDC)/European Control Conference (ECC) 2011 - Hilton Orlando Bonnet Creek, Orlando, United States of America Duration: 12 Dec 2011 → 15 Dec 2011 Conference number: 50th http://www.ieeecss.org/CAB/conferences/cdcecc2011/cfp.php https://www.ieee.org/conferences_events/conferences/conferencedetails/index.html?Conf_ID=15803 |

### Conference

Conference | IEEE Conference of Decision and Control (CDC)/European Control Conference (ECC) 2011 |
---|---|

Abbreviated title | CDC-ECC 2011 |

Country | United States of America |

City | Orlando |

Period | 12/12/11 → 15/12/11 |

Other | 2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011 |

Internet address |

## Cite this

Nazin, A., & Miller, B. (2011). The mirror descent control algorithm for weakly regular homogeneous finite Markov chains with unknown mean losses. In M. Polycarpou (Ed.),

*Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC)*(pp. 1779 - 1783). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/CDC.2011.6161477