The mirror descent control algorithm for weakly regular homogeneous finite Markov chains with unknown mean losses

Alexander Nazin, Boris Miller

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

2 Citations (Scopus)

Abstract

We address the adaptive stochastic control problem for a discrete time system described by controlled Markov chain with finite number of states. The mirror descent randomized control algorithm on the class of controlled homogeneous finite Markov chains with unknown mean losses has been proposed and studied. Here we develop the approach represented in Nazin and Miller (2011). The main assumptions are the following: processes are independent and stationary, nonnegative random losses are almost surely bounded by a given constant, and the connectivity assumption for the controlled Markov chain holds. The uncertainty is that the mean loss matrix is unknown. The novelty of the approach is in extension of the class of controlled homogeneous finite Markov chains to the chains with connectivity assumption. The main result consists in demonstration of the asymptotical upper bound (that is asymptotic by time) and in determining the explicit constant which is weakly depending on the logarithm of the number of states.
Original languageEnglish
Title of host publicationProceedings of the 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC)
EditorsMarios Polycarpou
Place of PublicationUSA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages1779 - 1783
Number of pages5
DOIs
Publication statusPublished - 2011
EventIEEE Conference of Decision and Control (CDC)/European Control Conference (ECC) 2011 - Hilton Orlando Bonnet Creek, Orlando, United States of America
Duration: 12 Dec 201115 Dec 2011
Conference number: 50th
http://www.ieeecss.org/CAB/conferences/cdcecc2011/cfp.php
https://www.ieee.org/conferences_events/conferences/conferencedetails/index.html?Conf_ID=15803

Conference

ConferenceIEEE Conference of Decision and Control (CDC)/European Control Conference (ECC) 2011
Abbreviated titleCDC-ECC 2011
CountryUnited States of America
CityOrlando
Period12/12/1115/12/11
Other2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011
Internet address

Cite this

Nazin, A., & Miller, B. (2011). The mirror descent control algorithm for weakly regular homogeneous finite Markov chains with unknown mean losses. In M. Polycarpou (Ed.), Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC) (pp. 1779 - 1783). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/CDC.2011.6161477