Control of Markov chains with constraints

Boris Miller, Gregory Miller, Konstantin Siemenikhin

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

5 Citations (Scopus)


The problem of the optimal control of a Markov chain with a finite state space is considered. We consider a non-stationary finite horizon problem with constraints, which are given as a set of inequalities. It is shown that if the optimal solution exists it satisfies the maximum principle and can be found as a solution of some deterministic optimal control problem. We suggest an approach to the numerical solution of the optimal control problem and demonstrate the equivalence of the constraint optimization problem to the minimax problem. The approach has been illustrated by a set of numerical examples related with queueing systems.
Original languageEnglish
Title of host publicationProceedings of the VIII International Conference on System Identification and Control Problems
EditorsV A Lototsky
Place of PublicationMoscow Russia
PublisherM A I K Nauka - Interperiodica
Pages1 - 23
Number of pages23
ISBN (Print)978591450242
Publication statusPublished - 2009
EventInternational Conference on System Identification and Control Problems (SICPRO) 2009 - Moscow Russia, Moscow Russia
Duration: 1 Jan 2009 → …


ConferenceInternational Conference on System Identification and Control Problems (SICPRO) 2009
CityMoscow Russia
Period1/01/09 → …

Cite this