Control of Markov chains with constraints

Boris Miller; Gregory Miller; Konstantin Siemenikhin

Control of Markov chains with constraints

Boris Miller, Gregory Miller, Konstantin Siemenikhin

School of Mathematics

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

5 Citations (Scopus)

Abstract

The problem of the optimal control of a Markov chain with a finite state space is considered. We consider a non-stationary finite horizon problem with constraints, which are given as a set of inequalities. It is shown that if the optimal solution exists it satisfies the maximum principle and can be found as a solution of some deterministic optimal control problem. We suggest an approach to the numerical solution of the optimal control problem and demonstrate the equivalence of the constraint optimization problem to the minimax problem. The approach has been illustrated by a set of numerical examples related with queueing systems.

Original language	English
Title of host publication	Proceedings of the VIII International Conference on System Identification and Control Problems
Editors	V A Lototsky
Place of Publication	Moscow Russia
Publisher	M A I K Nauka - Interperiodica
Pages	1 - 23
Number of pages	23
ISBN (Print)	978591450242
Publication status	Published - 2009
Event	International Conference on System Identification and Control Problems (SICPRO) 2009 - Moscow Russia, Moscow Russia Duration: 1 Jan 2009 → …

Conference

Conference	International Conference on System Identification and Control Problems (SICPRO) 2009
City	Moscow Russia
Period	1/01/09 → …

Access to Document

RM sElocation

Cite this

@inproceedings{4515d58e62174e20b69d1e8ec938930f,

title = "Control of Markov chains with constraints",

abstract = "The problem of the optimal control of a Markov chain with a finite state space is considered. We consider a non-stationary finite horizon problem with constraints, which are given as a set of inequalities. It is shown that if the optimal solution exists it satisfies the maximum principle and can be found as a solution of some deterministic optimal control problem. We suggest an approach to the numerical solution of the optimal control problem and demonstrate the equivalence of the constraint optimization problem to the minimax problem. The approach has been illustrated by a set of numerical examples related with queueing systems.",

author = "Boris Miller and Gregory Miller and Konstantin Siemenikhin",

year = "2009",

language = "English",

isbn = "978591450242",

pages = "1 -- 23",

editor = "Lototsky, {V A}",

booktitle = "Proceedings of the VIII International Conference on System Identification and Control Problems",

publisher = "M A I K Nauka - Interperiodica",

address = "Russian Federation",

note = "International Conference on System Identification and Control Problems (SICPRO) 2009 ; Conference date: 01-01-2009",

}

Miller, B, Miller, G & Siemenikhin, K 2009, Control of Markov chains with constraints. in VA Lototsky (ed.), Proceedings of the VIII International Conference on System Identification and Control Problems. M A I K Nauka - Interperiodica, Moscow Russia, pp. 1 - 23, International Conference on System Identification and Control Problems (SICPRO) 2009, Moscow Russia, 1/01/09.

Control of Markov chains with constraints. / Miller, Boris; Miller, Gregory; Siemenikhin, Konstantin.

Proceedings of the VIII International Conference on System Identification and Control Problems. ed. / V A Lototsky. Moscow Russia : M A I K Nauka - Interperiodica, 2009. p. 1 - 23.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - Control of Markov chains with constraints

AU - Miller, Boris

AU - Miller, Gregory

AU - Siemenikhin, Konstantin

PY - 2009

Y1 - 2009

N2 - The problem of the optimal control of a Markov chain with a finite state space is considered. We consider a non-stationary finite horizon problem with constraints, which are given as a set of inequalities. It is shown that if the optimal solution exists it satisfies the maximum principle and can be found as a solution of some deterministic optimal control problem. We suggest an approach to the numerical solution of the optimal control problem and demonstrate the equivalence of the constraint optimization problem to the minimax problem. The approach has been illustrated by a set of numerical examples related with queueing systems.

AB - The problem of the optimal control of a Markov chain with a finite state space is considered. We consider a non-stationary finite horizon problem with constraints, which are given as a set of inequalities. It is shown that if the optimal solution exists it satisfies the maximum principle and can be found as a solution of some deterministic optimal control problem. We suggest an approach to the numerical solution of the optimal control problem and demonstrate the equivalence of the constraint optimization problem to the minimax problem. The approach has been illustrated by a set of numerical examples related with queueing systems.

UR - http://www.sicpro.org/sicpro09/code/e09_01.htm

M3 - Conference Paper

SN - 978591450242

SP - 1

EP - 23

BT - Proceedings of the VIII International Conference on System Identification and Control Problems

A2 - Lototsky, V A

PB - M A I K Nauka - Interperiodica

CY - Moscow Russia

T2 - International Conference on System Identification and Control Problems (SICPRO) 2009

Y2 - 1 January 2009

ER -