ERA: Expert Retrieval and Assembly for early action prediction

Lin Geng Foo, Tianjiao Li, Hossein Rahmani, Qiuhong Ke, Jun Liu

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review


Early action prediction aims to successfully predict the class label of an action before it is completely performed. This is a challenging task because the beginning stages of different actions can be very similar, with only minor subtle differences for discrimination. In this paper, we propose a novel Expert Retrieval and Assembly (ERA) module that retrieves and assembles a set of experts most specialized at using discriminative subtle differences, to distinguish an input sample from other highly similar samples. To encourage our model to effectively use subtle differences for early action prediction, we push experts to discriminate exclusively between samples that are highly similar, forcing these experts to learn to use subtle differences that exist between those samples. Additionally, we design an effective Expert Learning Rate Optimization method that balances the experts’ optimization and leads to better performance. We evaluate our ERA module on four public action datasets and achieve state-of-the-art performance.
Original languageEnglish
Title of host publication17th European Conference Tel Aviv, Israel, October 23–27, 2022 Proceedings, Part XXXIV
EditorsShai Avidan, Gabriel Brostow, Moustapha Cisse, Giovanni Maria Farinella, Tal Hassner
Place of PublicationCham Switzerland
Number of pages19
ISBN (Electronic)9783031198304
ISBN (Print)9783031198298
Publication statusPublished - 2022
EventEuropean Conference on Computer Vision 2022 - Tel Aviv, Israel
Duration: 23 Oct 202227 Oct 2022
Conference number: 17th (Proceedings) (Website)


ConferenceEuropean Conference on Computer Vision 2022
Abbreviated titleECCV 2022
CityTel Aviv
Internet address


  • Early action prediction
  • Dynamic networks
  • Expert retrieval

Cite this