Best-response planning of thermostatically controlled loads under power constraints

Frits De Nijs, Matthijs T.J. Spaan, Mathys M. De Weerdt

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

13 Citations (Scopus)


Renewable power sources such as wind and solar are inflexible in their energy production, which requires demand to rapidly follow supply in order to maintain energy balance. Promising controllable demands are air-conditioners and heat pumps which use electric energy to maintain a temperature at a setpoint. Such Thermostatically Controlled Loads (TCLs) have been shown to be able to follow a power curve using reactive control. In this paper we investigate the use of planning under uncertainty to pro-actively control an aggregation of TCLs to overcome temporary grid imbalance. We present a formal definition of the planning problem under consideration, which we model using the Multi-Agent Markov Decision Process (MMDP) framework. Since we are dealing with hundreds of agents, solving the resulting MMDPs directly is intractable. Instead, we propose to decompose the problem by decoupling the interactions through arbitrage. Decomposition of the problem means relaxing the joint power consumption constraint, which means that joining the plans together can cause overconsumption. Arbitrage acts as a conflict resolution mechanism during policy execution, using the future expected value of policies to determine which TCLs should receive the available energy. We experimentally compare several methods to plan with arbitrage, and conclude that a best response-like mechanism is a scalable approach that returns near-optimal solutions.

Original languageEnglish
Title of host publicationProceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence
EditorsBlai Bonet, Sven Koenig
Place of PublicationPalo Alto CA USA
PublisherAssociation for the Advancement of Artificial Intelligence (AAAI)
Number of pages7
ISBN (Print)9781577356998
Publication statusPublished - 1 Jun 2015
Externally publishedYes
EventAAAI Conference on Artificial Intelligence 2015 - Hyatt Regency, Austin, United States of America
Duration: 25 Jan 201530 Jan 2015
Conference number: 29th


ConferenceAAAI Conference on Artificial Intelligence 2015
Abbreviated titleAAAI 2015
Country/TerritoryUnited States of America
OtherCo-located with the 27th Innovative Applications of Artificial Intelligence Conference. Papers at the AAAI 2015 conference will be related here. Any papers presented at the IAAI 2015 part of the conference will be related to that event. The two conferences should have a "relation" to each other put in place to recognise that the conferences were combined into one proceedings.
Internet address

Cite this