Background subtraction based on Deep Pixel Distribution Learning

Chenqiu Zhao, Tat-Len Cham, Xinyu Ren, Jianfei Cai, Haichen Zhu

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

6 Citations (Scopus)


Previous approaches to background subtraction typically address the problem by formulating a representation of the background, and comparing the background to new frames. In this work, we focus on the essence of background subtraction, which is the classification of a pixel's current observation in comparison to historical observations, and propose a Deep Pixel Distribution Learning (DPDL) model for background subtraction. In the DPDL model, a novel pixel-based feature, called the Random Permutation of Temporal Pixels (RPoTP), is used to represent the distribution of past observations for a particular pixel, in which the temporal correlation between observations is deliberately obfuscated. Subsequently a convolutional neural network (CNN) is used to learn the distribution for determining whether the current observation is foreground or background, with the random permutation enabling the framework to focus primarily on the distribution of observations, rather than be misled by learning spurious temporal correlations. In addition, the pixel-wise representation allows for a large number of RPoTP features to be captured even with a limited number of groundtruth frames, with the DPDL model being effective even with only a single groundtruth frame. The proposed framework is able to achieve promising results in diverse natural scenes, and a comprehensive evaluation on standard benchmarks demonstrates the superiority of our work to state-of-the-art methods. The source code ispublicly available at

Original languageEnglish
Title of host publicationConference Proceedings
Subtitle of host publication2018 IEEE International Conference on Multimedia and Expo (ICME)
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages6
ISBN (Electronic)9781538617373, 9781538617366
ISBN (Print)9781538617380
Publication statusPublished - 2018
Externally publishedYes
EventIEEE International Conference on Multimedia and Expo 2018 - San Diego, United States of America
Duration: 23 Jul 201827 Jul 2018


ConferenceIEEE International Conference on Multimedia and Expo 2018
Abbreviated titleICME 2018
CountryUnited States of America
CitySan Diego
Internet address


  • Background subtraction
  • deep learning
  • motion detection
  • pixel distribution
  • random permutation

Cite this