The mirror descent control algorithm for weakly regular homogeneous finite Markov chains with unknown mean losses

Alexander Nazin; Boris Miller

doi:10.1109/CDC.2011.6161477

The mirror descent control algorithm for weakly regular homogeneous finite Markov chains with unknown mean losses

Alexander Nazin, Boris Miller

School of Mathematics

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

2 Citations (Scopus)

Abstract

We address the adaptive stochastic control problem for a discrete time system described by controlled Markov chain with finite number of states. The mirror descent randomized control algorithm on the class of controlled homogeneous finite Markov chains with unknown mean losses has been proposed and studied. Here we develop the approach represented in Nazin and Miller (2011). The main assumptions are the following: processes are independent and stationary, nonnegative random losses are almost surely bounded by a given constant, and the connectivity assumption for the controlled Markov chain holds. The uncertainty is that the mean loss matrix is unknown. The novelty of the approach is in extension of the class of controlled homogeneous finite Markov chains to the chains with connectivity assumption. The main result consists in demonstration of the asymptotical upper bound (that is asymptotic by time) and in determining the explicit constant which is weakly depending on the logarithm of the number of states.

Original language	English
Title of host publication	Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC)
Editors	Marios Polycarpou
Place of Publication	USA
Publisher	IEEE, Institute of Electrical and Electronics Engineers
Pages	1779 - 1783
Number of pages	5
DOIs	https://doi.org/10.1109/CDC.2011.6161477
Publication status	Published - 2011
Event	IEEE Conference of Decision and Control (CDC)/European Control Conference (ECC) 2011 - Hilton Orlando Bonnet Creek, Orlando, United States of America Duration: 12 Dec 2011 → 15 Dec 2011 Conference number: 50th http://www.ieeecss.org/CAB/conferences/cdcecc2011/cfp.php https://www.ieee.org/conferences_events/conferences/conferencedetails/index.html?Conf_ID=15803

Conference

Conference	IEEE Conference of Decision and Control (CDC)/European Control Conference (ECC) 2011
Abbreviated title	CDC-ECC 2011
Country	United States of America
City	Orlando
Period	12/12/11 → 15/12/11
Other	2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011
Internet address	http://www.ieeecss.org/CAB/conferences/cdcecc2011/cfp.php https://www.ieee.org/conferences_events/conferences/conferencedetails/index.html?Conf_ID=15803

Access to Document

10.1109/CDC.2011.6161477

RM sElocation

Cite this

Nazin, A., & Miller, B. (2011). The mirror descent control algorithm for weakly regular homogeneous finite Markov chains with unknown mean losses. In M. Polycarpou (Ed.), Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC) (pp. 1779 - 1783). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/CDC.2011.6161477

Nazin, Alexander ; Miller, Boris. / The mirror descent control algorithm for weakly regular homogeneous finite Markov chains with unknown mean losses. Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC). editor / Marios Polycarpou. USA : IEEE, Institute of Electrical and Electronics Engineers, 2011. pp. 1779 - 1783

@inproceedings{27c982fbe7fa4d54aa1e2f824a4abb68,

title = "The mirror descent control algorithm for weakly regular homogeneous finite Markov chains with unknown mean losses",

abstract = "We address the adaptive stochastic control problem for a discrete time system described by controlled Markov chain with finite number of states. The mirror descent randomized control algorithm on the class of controlled homogeneous finite Markov chains with unknown mean losses has been proposed and studied. Here we develop the approach represented in Nazin and Miller (2011). The main assumptions are the following: processes are independent and stationary, nonnegative random losses are almost surely bounded by a given constant, and the connectivity assumption for the controlled Markov chain holds. The uncertainty is that the mean loss matrix is unknown. The novelty of the approach is in extension of the class of controlled homogeneous finite Markov chains to the chains with connectivity assumption. The main result consists in demonstration of the asymptotical upper bound (that is asymptotic by time) and in determining the explicit constant which is weakly depending on the logarithm of the number of states.",

author = "Alexander Nazin and Boris Miller",

year = "2011",

doi = "10.1109/CDC.2011.6161477",

language = "English",

pages = "1779 -- 1783",

editor = "Marios Polycarpou",

booktitle = "Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC)",

publisher = "IEEE, Institute of Electrical and Electronics Engineers",

address = "United States of America",

note = "IEEE Conference of Decision and Control (CDC)/European Control Conference (ECC) 2011, CDC-ECC 2011 ; Conference date: 12-12-2011 Through 15-12-2011",

url = "http://www.ieeecss.org/CAB/conferences/cdcecc2011/cfp.php, https://www.ieee.org/conferences_events/conferences/conferencedetails/index.html?Conf_ID=15803",

}

Nazin, A & Miller, B 2011, The mirror descent control algorithm for weakly regular homogeneous finite Markov chains with unknown mean losses. in M Polycarpou (ed.), Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC). IEEE, Institute of Electrical and Electronics Engineers, USA, pp. 1779 - 1783, IEEE Conference of Decision and Control (CDC)/European Control Conference (ECC) 2011, Orlando, United States of America, 12/12/11. https://doi.org/10.1109/CDC.2011.6161477

The mirror descent control algorithm for weakly regular homogeneous finite Markov chains with unknown mean losses. / Nazin, Alexander; Miller, Boris.

Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC). ed. / Marios Polycarpou. USA : IEEE, Institute of Electrical and Electronics Engineers, 2011. p. 1779 - 1783.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - The mirror descent control algorithm for weakly regular homogeneous finite Markov chains with unknown mean losses

AU - Nazin, Alexander

AU - Miller, Boris

N1 - Conference code: 50th

PY - 2011

Y1 - 2011

N2 - We address the adaptive stochastic control problem for a discrete time system described by controlled Markov chain with finite number of states. The mirror descent randomized control algorithm on the class of controlled homogeneous finite Markov chains with unknown mean losses has been proposed and studied. Here we develop the approach represented in Nazin and Miller (2011). The main assumptions are the following: processes are independent and stationary, nonnegative random losses are almost surely bounded by a given constant, and the connectivity assumption for the controlled Markov chain holds. The uncertainty is that the mean loss matrix is unknown. The novelty of the approach is in extension of the class of controlled homogeneous finite Markov chains to the chains with connectivity assumption. The main result consists in demonstration of the asymptotical upper bound (that is asymptotic by time) and in determining the explicit constant which is weakly depending on the logarithm of the number of states.

AB - We address the adaptive stochastic control problem for a discrete time system described by controlled Markov chain with finite number of states. The mirror descent randomized control algorithm on the class of controlled homogeneous finite Markov chains with unknown mean losses has been proposed and studied. Here we develop the approach represented in Nazin and Miller (2011). The main assumptions are the following: processes are independent and stationary, nonnegative random losses are almost surely bounded by a given constant, and the connectivity assumption for the controlled Markov chain holds. The uncertainty is that the mean loss matrix is unknown. The novelty of the approach is in extension of the class of controlled homogeneous finite Markov chains to the chains with connectivity assumption. The main result consists in demonstration of the asymptotical upper bound (that is asymptotic by time) and in determining the explicit constant which is weakly depending on the logarithm of the number of states.

UR - http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6161477

U2 - 10.1109/CDC.2011.6161477

DO - 10.1109/CDC.2011.6161477

M3 - Conference Paper

SP - 1779

EP - 1783

BT - Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC)

A2 - Polycarpou, Marios

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - USA

T2 - IEEE Conference of Decision and Control (CDC)/European Control Conference (ECC) 2011

Y2 - 12 December 2011 through 15 December 2011

ER -

Nazin A, Miller B. The mirror descent control algorithm for weakly regular homogeneous finite Markov chains with unknown mean losses. In Polycarpou M, editor, Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC). USA: IEEE, Institute of Electrical and Electronics Engineers. 2011. p. 1779 - 1783 https://doi.org/10.1109/CDC.2011.6161477