Meta learning with differentiable closed-form solver for fast video object segmentation

Yu Liu, Lingqiao Liu, Haokui Zhang, Hamid Rezatofighi, Qingsen Yan, Ian Reid

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review


Video object segmentation plays a vital role to many robotic tasks, beyond the satisfied accuracy, quickly adapt to the new scenario with very limited annotations and conduct a quick inference are also important. In this paper, we are specifically concerned with the task of fast segmenting all pixels of a target object in all frames, given the annotation mask in the first frame. Even when such annotation is available, this remains a challenging problem because of the changing appearance and shape of the object over time. In this paper, we tackle this task by formulating it as a meta-learning problem, where the base learner grasping the semantic scene understanding for a general type of objects, and the meta learner quickly adapting the appearance of the target object with a few examples. Our proposed meta-learning method uses a closed form optimizer, the so-called "ridge regression", which has been shown to be conducive for fast and better training convergence. Moreover, we propose a mechanism, named "block splitting", to further speed up the training process as well as to reduce the number of learning parameters. In comparison with the state-of-the art methods, our proposed framework achieves significant boost up in processing speed, while having highly comparable performance compared to the best performing methods on the widely used datasets. Video demo can be found here 1.

Original languageEnglish
Title of host publicationIEEE International Workshop on Intelligent Robots and Systems (IROS)
EditorsHyunglae Lee
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages8
ISBN (Electronic)9781728162126
ISBN (Print)9781728162133
Publication statusPublished - 2020
Externally publishedYes
EventIEEE/RSJ International Conference on Intelligent Robots and Systems 2020 - Virtual, Las Vegas, United States of America
Duration: 24 Jan 202124 Jan 2021 (Website)

Publication series

NameIEEE International Conference on Intelligent Robots and Systems
PublisherIEEE, Institute of Electrical and Electronics Engineers
ISSN (Print)2153-0858
ISSN (Electronic)2153-0866


ConferenceIEEE/RSJ International Conference on Intelligent Robots and Systems 2020
Abbreviated titleIROS 2020
CountryUnited States of America
CityLas Vegas
Internet address

Cite this