Towards tensor representation of controlled coupled markov chains

Daniel McInnes, Boris Miller, Gregory Miller, Sergei Schreider

Research output: Contribution to journalArticleResearchpeer-review

1 Citation (Scopus)


For a controlled system of coupled Markov chains, which share common control parameters, a tensor description is proposed. A control optimality condition in the form of a dynamic programming equation is derived in tensor form. This condition can be reduced to a system of coupled ordinary differential equations and admits an effective numerical solution. As an application example, the problem of the optimal control for a system of water reservoirs with phase and balance constraints is considered.

Original languageEnglish
Article number1712
Number of pages17
Issue number10
Publication statusPublished - 5 Oct 2020


  • Coupled markov chains
  • Dynamic programming
  • Optimal control
  • Stochastic control
  • Tensor representation

Cite this