Management of a large dam via optimal price control

This paper considers the optimal management of a dam with its level approximated by N discrete states and modeled via a continuous-time controlled Markov chain on a finite time horizon. The inflow process is seasonal, and so non-stationary, as are the customer demands. The control takes the form of price, the bounds of which are set by regulators, and we take into account the individual sectoral demands of customers. The general approach to the solution of this problem is to consider the stochastic optimization problem in the average sense and solve it via dynamic programming. We show that an optimal price can be obtained for each state based on minimizing the difference between desired and optimal consumption, and minimizing the probability that the dam level falls below a prescribed level both on the control period and at the terminal time. The result is illustrated by a numerical example.