Multi-modal unsupervised feature learning for RGB-D scene labeling

Anran Wang; Jiwen Lu; Gang Wang; Jianfei Cai; Tat-Jen Cham

doi:10.1007/978-3-319-10602-1_30

Multi-modal unsupervised feature learning for RGB-D scene labeling

Anran Wang, Jiwen Lu, Gang Wang, Jianfei Cai, Tat-Jen Cham

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

39 Citations (Scopus)

Abstract

Most of the existing approaches for RGB-D indoor scene labeling employ hand-crafted features for each modality independently and combine them in a heuristic manner. There has been some attempt on directly learning features from raw RGB-D data, but the performance is not satisfactory. In this paper, we adapt the unsupervised feature learning technique for RGB-D labeling as a multi-modality learning problem. Our learning framework performs feature learning and feature encoding simultaneously which significantly boosts the performance. By stacking basic learning structure, higher-level features are derived and combined with lower-level features for better representing RGB-D data. Experimental results on the benchmark NYU depth dataset show that our method achieves competitive performance, compared with state-of-the-art.

Original language	English
Title of host publication	Computer Vision – ECCV 2014
Subtitle of host publication	13th European Conference Zurich, Switzerland, September 6-12, 2014 Proceedings, Part V
Editors	David Fleet, Tomas Pajdla, Bernt Schiele, Tinne Tuytelaars
Place of Publication	Cham Switzerland
Publisher	Springer
Pages	453-467
Number of pages	15
ISBN (Electronic)	9783319106021
ISBN (Print)	9783319106014
DOIs	https://doi.org/10.1007/978-3-319-10602-1_30
Publication status	Published - 2014
Externally published	Yes
Event	European Conference on Computer Vision 2014 - Zurich, Switzerland Duration: 6 Sept 2014 → 12 Sept 2014 Conference number: 13th http://eccv2014.org/ https://link.springer.com/book/10.1007/978-3-319-10590-1 (Proceedings)

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer
Number	5
Volume	8693
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	European Conference on Computer Vision 2014
Abbreviated title	ECCV 2014
Country/Territory	Switzerland
City	Zurich
Period	6/09/14 → 12/09/14
Internet address	http://eccv2014.org/ https://link.springer.com/book/10.1007/978-3-319-10590-1 (Proceedings)

Keywords

joint feature learning and encoding
multi-modality
RGB-D scene labeling
unsupervised feature learning

Access to Document

10.1007/978-3-319-10602-1_30

Cite this

Wang, A., Lu, J., Wang, G., Cai, J., & Cham, T.-J. (2014). Multi-modal unsupervised feature learning for RGB-D scene labeling. In D. Fleet, T. Pajdla, B. Schiele, & T. Tuytelaars (Eds.), Computer Vision – ECCV 2014: 13th European Conference Zurich, Switzerland, September 6-12, 2014 Proceedings, Part V (pp. 453-467). (Lecture Notes in Computer Science ; Vol. 8693, No. 5). Springer. https://doi.org/10.1007/978-3-319-10602-1_30

Wang, Anran ; Lu, Jiwen ; Wang, Gang et al. / Multi-modal unsupervised feature learning for RGB-D scene labeling. Computer Vision – ECCV 2014: 13th European Conference Zurich, Switzerland, September 6-12, 2014 Proceedings, Part V. editor / David Fleet ; Tomas Pajdla ; Bernt Schiele ; Tinne Tuytelaars. Cham Switzerland : Springer, 2014. pp. 453-467 (Lecture Notes in Computer Science ; 5).

@inproceedings{e80074fdb0024c8cb28c14fc8becc967,

title = "Multi-modal unsupervised feature learning for RGB-D scene labeling",

abstract = "Most of the existing approaches for RGB-D indoor scene labeling employ hand-crafted features for each modality independently and combine them in a heuristic manner. There has been some attempt on directly learning features from raw RGB-D data, but the performance is not satisfactory. In this paper, we adapt the unsupervised feature learning technique for RGB-D labeling as a multi-modality learning problem. Our learning framework performs feature learning and feature encoding simultaneously which significantly boosts the performance. By stacking basic learning structure, higher-level features are derived and combined with lower-level features for better representing RGB-D data. Experimental results on the benchmark NYU depth dataset show that our method achieves competitive performance, compared with state-of-the-art.",

keywords = "joint feature learning and encoding, multi-modality, RGB-D scene labeling, unsupervised feature learning",

author = "Anran Wang and Jiwen Lu and Gang Wang and Jianfei Cai and Tat-Jen Cham",

year = "2014",

doi = "10.1007/978-3-319-10602-1_30",

language = "English",

isbn = "9783319106014",

series = "Lecture Notes in Computer Science ",

publisher = "Springer",

number = "5",

pages = "453--467",

editor = "David Fleet and Pajdla, {Tomas } and Bernt Schiele and Tuytelaars, {Tinne }",

booktitle = "Computer Vision – ECCV 2014",

note = "European Conference on Computer Vision 2014, ECCV 2014 ; Conference date: 06-09-2014 Through 12-09-2014",

url = "http://eccv2014.org/, https://link.springer.com/book/10.1007/978-3-319-10590-1",

}

Wang, A, Lu, J, Wang, G, Cai, J & Cham, T-J 2014, Multi-modal unsupervised feature learning for RGB-D scene labeling. in D Fleet, T Pajdla, B Schiele & T Tuytelaars (eds), Computer Vision – ECCV 2014: 13th European Conference Zurich, Switzerland, September 6-12, 2014 Proceedings, Part V. Lecture Notes in Computer Science , no. 5, vol. 8693, Springer, Cham Switzerland, pp. 453-467, European Conference on Computer Vision 2014, Zurich, Switzerland, 6/09/14. https://doi.org/10.1007/978-3-319-10602-1_30

Multi-modal unsupervised feature learning for RGB-D scene labeling. / Wang, Anran; Lu, Jiwen; Wang, Gang et al.
Computer Vision – ECCV 2014: 13th European Conference Zurich, Switzerland, September 6-12, 2014 Proceedings, Part V. ed. / David Fleet; Tomas Pajdla; Bernt Schiele; Tinne Tuytelaars. Cham Switzerland: Springer, 2014. p. 453-467 (Lecture Notes in Computer Science ; Vol. 8693, No. 5).

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - Multi-modal unsupervised feature learning for RGB-D scene labeling

AU - Wang, Anran

AU - Lu, Jiwen

AU - Wang, Gang

AU - Cai, Jianfei

AU - Cham, Tat-Jen

N1 - Conference code: 13th

PY - 2014

Y1 - 2014

N2 - Most of the existing approaches for RGB-D indoor scene labeling employ hand-crafted features for each modality independently and combine them in a heuristic manner. There has been some attempt on directly learning features from raw RGB-D data, but the performance is not satisfactory. In this paper, we adapt the unsupervised feature learning technique for RGB-D labeling as a multi-modality learning problem. Our learning framework performs feature learning and feature encoding simultaneously which significantly boosts the performance. By stacking basic learning structure, higher-level features are derived and combined with lower-level features for better representing RGB-D data. Experimental results on the benchmark NYU depth dataset show that our method achieves competitive performance, compared with state-of-the-art.

AB - Most of the existing approaches for RGB-D indoor scene labeling employ hand-crafted features for each modality independently and combine them in a heuristic manner. There has been some attempt on directly learning features from raw RGB-D data, but the performance is not satisfactory. In this paper, we adapt the unsupervised feature learning technique for RGB-D labeling as a multi-modality learning problem. Our learning framework performs feature learning and feature encoding simultaneously which significantly boosts the performance. By stacking basic learning structure, higher-level features are derived and combined with lower-level features for better representing RGB-D data. Experimental results on the benchmark NYU depth dataset show that our method achieves competitive performance, compared with state-of-the-art.

KW - joint feature learning and encoding

KW - multi-modality

KW - RGB-D scene labeling

KW - unsupervised feature learning

UR - http://www.scopus.com/inward/record.url?scp=84906490580&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-10602-1_30

DO - 10.1007/978-3-319-10602-1_30

M3 - Conference Paper

AN - SCOPUS:84906490580

SN - 9783319106014

T3 - Lecture Notes in Computer Science

SP - 453

EP - 467

BT - Computer Vision – ECCV 2014

A2 - Fleet, David

A2 - Pajdla, Tomas

A2 - Schiele, Bernt

A2 - Tuytelaars, Tinne

PB - Springer

CY - Cham Switzerland

T2 - European Conference on Computer Vision 2014

Y2 - 6 September 2014 through 12 September 2014

ER -

Wang A, Lu J, Wang G, Cai J, Cham TJ. Multi-modal unsupervised feature learning for RGB-D scene labeling. In Fleet D, Pajdla T, Schiele B, Tuytelaars T, editors, Computer Vision – ECCV 2014: 13th European Conference Zurich, Switzerland, September 6-12, 2014 Proceedings, Part V. Cham Switzerland: Springer. 2014. p. 453-467. (Lecture Notes in Computer Science ; 5). doi: 10.1007/978-3-319-10602-1_30

Multi-modal unsupervised feature learning for RGB-D scene labeling

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Cite this