Towards tensor representation of controlled coupled markov chains

Daniel McInnes, Boris Miller, Gregory Miller, Sergei Schreider

Research output: Contribution to journalArticleResearchpeer-review

Abstract

For a controlled system of coupled Markov chains, which share common control parameters, a tensor description is proposed. A control optimality condition in the form of a dynamic programming equation is derived in tensor form. This condition can be reduced to a system of coupled ordinary differential equations and admits an effective numerical solution. As an application example, the problem of the optimal control for a system of water reservoirs with phase and balance constraints is considered.

Original languageEnglish
Article number1712
Number of pages17
JournalMathematics
Volume8
Issue number10
DOIs
Publication statusPublished - 5 Oct 2020

Keywords

  • Coupled markov chains
  • Dynamic programming
  • Optimal control
  • Stochastic control
  • Tensor representation

Cite this