Regularization in hierarchical time series forecasting with application to electricity smart meter data

Souhaib Ben Taieb, Jiafan Yu, Mateus Neves Barreto, Ram Rajagopal

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review


Accurate electricity demand forecast plays a key role in sustainable power systems. It enables better decision making in the planning of electricity generation and distribution for many use cases. The electricity demand data can often be represented in a hierarchical structure. For example, the electricity consumption of a whole country could be disaggregated by states, cities, and households. Hierarchical forecasts require not only good prediction accuracy at each level of the hierarchy, but also the consistency between different levels. State-of-the-art hierarchical forecasting methods usually apply adjustments on the individual level forecasts to satisfy the aggregation constraints. However, the high-dimensionality of the unpenalized regression problem and the estimation errors in the high-dimensional error covariance matrix can lead to increased variability in the revised forecasts with poor prediction performance. In order to provide more robustness to estimation errors in the adjustments, we present a new hierarchical forecasting algorithm that computes sparse adjustments while still preserving the aggregation constraints. We formulate the problem as a high-dimensional penalized regression, which can be efficiently solved using cyclical coordinate descent methods. We also conduct experiments using a large-scale hierarchical electricity demand data. The results confirm the effectiveness of our approach compared to state-of-the-art hierarchical forecasting methods, in both the sparsity of the adjustments and the prediction accuracy. The proposed approach to hierarchical forecasting could be useful for energy generation including solar and wind energy, as well as numerous other applications.
Original languageEnglish
Title of host publicationProceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17)
EditorsSatinder Singh, Shaul Markovitch
Place of PublicationPalto Alto CA USA
PublisherAssociation for the Advancement of Artificial Intelligence (AAAI)
Number of pages7
Publication statusPublished - 2017
EventAAAI Conference on Artificial Intelligence 2017 - Hilton San Francisco Union Square, San Francisco, United States of America
Duration: 4 Feb 201710 Feb 2017
Conference number: 31st


ConferenceAAAI Conference on Artificial Intelligence 2017
Abbreviated titleAAAI 2017
Country/TerritoryUnited States of America
CitySan Francisco
Internet address

Cite this