The importance of metric learning for robotic vision: open set recognition and active learning

Benjamin J. Meyer, Tom Drummond

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

State-of-the-art deep neural network recognition systems are designed for a static and closed world. It is usually assumed that the distribution at test time will be the same as the distribution during training. As a result, classifiers are forced to categorise observations into one out of a set of predefined semantic classes. Robotic problems are dynamic and open world; a robot will likely observe objects that are from outside of the training set distribution. Classifier outputs in robotic applications can lead to real-world robotic action and as such, a practical recognition system should not silently fail by confidently misclassifying novel observations. We show how a deep metric learning classification system can be applied to such open set recognition problems, allowing the classifier to label novel observations as unknown. Further to detecting novel examples, we propose an open set active learning approach that allows a robot to efficiently query a user about unknown observations. Our approach enables a robot to improve its understanding of the true distribution of data in the environment, from a small number of label queries. Experimental results show that our approach significantly outperforms comparable methods in both the open set recognition and active learning problems.

Original languageEnglish
Title of host publication2019 International Conference on Robotics and Automation (ICRA)
EditorsJaydev P. Desai
Place of PublicationDanvers MA USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages2924-2931
Number of pages8
ISBN (Electronic)9781538660263
DOIs
Publication statusPublished - 2019
EventIEEE International Conference on Robotics and Automation 2019 - Montreal, Canada
Duration: 20 May 201924 May 2019

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
PublisherIEEE, Institute of Electrical and Electronics Engineers
ISSN (Print)1050-4729

Conference

ConferenceIEEE International Conference on Robotics and Automation 2019
Abbreviated titleICRA 2019
CountryCanada
CityMontreal
Period20/05/1924/05/19

Cite this

Meyer, B. J., & Drummond, T. (2019). The importance of metric learning for robotic vision: open set recognition and active learning. In J. P. Desai (Ed.), 2019 International Conference on Robotics and Automation (ICRA) (pp. 2924-2931). (Proceedings - IEEE International Conference on Robotics and Automation). Danvers MA USA: IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICRA.2019.8794188
Meyer, Benjamin J. ; Drummond, Tom. / The importance of metric learning for robotic vision : open set recognition and active learning. 2019 International Conference on Robotics and Automation (ICRA). editor / Jaydev P. Desai. Danvers MA USA : IEEE, Institute of Electrical and Electronics Engineers, 2019. pp. 2924-2931 (Proceedings - IEEE International Conference on Robotics and Automation).
@inproceedings{1eb2ed8d78764e4798d0919cce109021,
title = "The importance of metric learning for robotic vision: open set recognition and active learning",
abstract = "State-of-the-art deep neural network recognition systems are designed for a static and closed world. It is usually assumed that the distribution at test time will be the same as the distribution during training. As a result, classifiers are forced to categorise observations into one out of a set of predefined semantic classes. Robotic problems are dynamic and open world; a robot will likely observe objects that are from outside of the training set distribution. Classifier outputs in robotic applications can lead to real-world robotic action and as such, a practical recognition system should not silently fail by confidently misclassifying novel observations. We show how a deep metric learning classification system can be applied to such open set recognition problems, allowing the classifier to label novel observations as unknown. Further to detecting novel examples, we propose an open set active learning approach that allows a robot to efficiently query a user about unknown observations. Our approach enables a robot to improve its understanding of the true distribution of data in the environment, from a small number of label queries. Experimental results show that our approach significantly outperforms comparable methods in both the open set recognition and active learning problems.",
author = "Meyer, {Benjamin J.} and Tom Drummond",
year = "2019",
doi = "10.1109/ICRA.2019.8794188",
language = "English",
series = "Proceedings - IEEE International Conference on Robotics and Automation",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
pages = "2924--2931",
editor = "Desai, {Jaydev P.}",
booktitle = "2019 International Conference on Robotics and Automation (ICRA)",
address = "United States of America",

}

Meyer, BJ & Drummond, T 2019, The importance of metric learning for robotic vision: open set recognition and active learning. in JP Desai (ed.), 2019 International Conference on Robotics and Automation (ICRA). Proceedings - IEEE International Conference on Robotics and Automation, IEEE, Institute of Electrical and Electronics Engineers, Danvers MA USA, pp. 2924-2931, IEEE International Conference on Robotics and Automation 2019, Montreal, Canada, 20/05/19. https://doi.org/10.1109/ICRA.2019.8794188

The importance of metric learning for robotic vision : open set recognition and active learning. / Meyer, Benjamin J.; Drummond, Tom.

2019 International Conference on Robotics and Automation (ICRA). ed. / Jaydev P. Desai. Danvers MA USA : IEEE, Institute of Electrical and Electronics Engineers, 2019. p. 2924-2931 (Proceedings - IEEE International Conference on Robotics and Automation).

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

TY - GEN

T1 - The importance of metric learning for robotic vision

T2 - open set recognition and active learning

AU - Meyer, Benjamin J.

AU - Drummond, Tom

PY - 2019

Y1 - 2019

N2 - State-of-the-art deep neural network recognition systems are designed for a static and closed world. It is usually assumed that the distribution at test time will be the same as the distribution during training. As a result, classifiers are forced to categorise observations into one out of a set of predefined semantic classes. Robotic problems are dynamic and open world; a robot will likely observe objects that are from outside of the training set distribution. Classifier outputs in robotic applications can lead to real-world robotic action and as such, a practical recognition system should not silently fail by confidently misclassifying novel observations. We show how a deep metric learning classification system can be applied to such open set recognition problems, allowing the classifier to label novel observations as unknown. Further to detecting novel examples, we propose an open set active learning approach that allows a robot to efficiently query a user about unknown observations. Our approach enables a robot to improve its understanding of the true distribution of data in the environment, from a small number of label queries. Experimental results show that our approach significantly outperforms comparable methods in both the open set recognition and active learning problems.

AB - State-of-the-art deep neural network recognition systems are designed for a static and closed world. It is usually assumed that the distribution at test time will be the same as the distribution during training. As a result, classifiers are forced to categorise observations into one out of a set of predefined semantic classes. Robotic problems are dynamic and open world; a robot will likely observe objects that are from outside of the training set distribution. Classifier outputs in robotic applications can lead to real-world robotic action and as such, a practical recognition system should not silently fail by confidently misclassifying novel observations. We show how a deep metric learning classification system can be applied to such open set recognition problems, allowing the classifier to label novel observations as unknown. Further to detecting novel examples, we propose an open set active learning approach that allows a robot to efficiently query a user about unknown observations. Our approach enables a robot to improve its understanding of the true distribution of data in the environment, from a small number of label queries. Experimental results show that our approach significantly outperforms comparable methods in both the open set recognition and active learning problems.

UR - http://www.scopus.com/inward/record.url?scp=85071481287&partnerID=8YFLogxK

U2 - 10.1109/ICRA.2019.8794188

DO - 10.1109/ICRA.2019.8794188

M3 - Conference Paper

AN - SCOPUS:85071481287

T3 - Proceedings - IEEE International Conference on Robotics and Automation

SP - 2924

EP - 2931

BT - 2019 International Conference on Robotics and Automation (ICRA)

A2 - Desai, Jaydev P.

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Danvers MA USA

ER -

Meyer BJ, Drummond T. The importance of metric learning for robotic vision: open set recognition and active learning. In Desai JP, editor, 2019 International Conference on Robotics and Automation (ICRA). Danvers MA USA: IEEE, Institute of Electrical and Electronics Engineers. 2019. p. 2924-2931. (Proceedings - IEEE International Conference on Robotics and Automation). https://doi.org/10.1109/ICRA.2019.8794188