Multiple Foreground Recognition and Cosegmentation: an object-oriented CRF model with robust higher-order potentials

Hongyuan Zhu, Jiangbo Lu, Jianfei Cai, Jianming Zheng, Nadia M. Thalmann

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

21 Citations (Scopus)


Localizing, recognizing, and segmenting multiple foreground objects jointly from a general user's photo stream that records a specific event is an important task with many useful applications. As argued in recent Multiple Foreground Cosegmentation (MFC) work by Kim and Xing, this task is very challenging in that it contrasts substantially from the classical cosegmentation problem, and aims to parse a set of realistic event photos but each containing irregularly occurring multiple foregrounds with high appearance and scene configuration variations. Inspired by the impressive advance in scene understanding and object recognition, this paper casts the multiple foreground recognition and cosegmentation (MFRC) problem within a conditional random fields (CRFs) framework in a principled manner. We capitalize centrally on the key objective that MFRC is to segment out and annotate foreground objects or 'things' rather than 'stuff'. To this end, we exploit a few complementary objectness cues (e.g. contours, object detectors and layout) and propose novel and efficient methods to capture object-level information. Integrating object potentials as soft constraints (e.g. robust higher-order potentials defined over detected object regions) with low-level unary and pairwise terms holistically, we solve the MFRC task with a probabilistic CRF model. The inference for such a CRF model is performed efficiently with graph cut based move making algorithms. With a minimal amount of user annotations on just a few example photos, the proposed approach produces spatially coherent, boundary-aligned segmentation results with correct and consistent object labeling. Experiments on the FlickrMFC dataset justify that our method achieves state-of-the-art performance.

Original languageEnglish
Title of host publication2014 IEEE Winter Conference on Applications of Computer Vision, WACV 2014
EditorsWalter Scheirer, Ruigang Yang, Charles Stewart
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages8
ISBN (Print)9781479949854
Publication statusPublished - 2014
Externally publishedYes
EventIEEE Winter Conference on Applications of Computer Vision 2014 - Steamboat Springs, United States of America
Duration: 24 Mar 201426 Mar 2014 (Proceedings)


ConferenceIEEE Winter Conference on Applications of Computer Vision 2014
Abbreviated titleWACV 2014
Country/TerritoryUnited States of America
CitySteamboat Springs
Internet address

Cite this