Subspace regularizers for few-shot class incremental learning

Afra Feyza Akyürek, Ekin Akyürek, Derry Tanti Wijaya, Jacob Andreas

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

13 Citations (Scopus)

Abstract

Few-shot class incremental learning-the problem of updating a trained classifier to discriminate among an expanded set of classes with limited labeled data-is a key challenge for machine learning systems deployed in non-stationary environments. Existing approaches to the problem rely on complex model architectures and training procedures that are difficult to tune and re-use. In this paper, we present an extremely simple approach that enables the use of ordinary logistic regression classifiers for few-shot incremental learning. The key to this approach is a new family of subspace regularization schemes that encourage weight vectors for new classes to lie close to the subspace spanned by the weights of existing classes. When combined with pretrained convolutional feature extractors, logistic regression models trained with subspace regularization outperform specialized, state-of-the-art approaches to few-shot incremental image classification by up to 23% on the miniImageNet dataset. Because of its simplicity, subspace regularization can be straightforwardly configured to incorporate additional background information about the new classes (including class names and descriptions specified in natural language); this offers additional control over the trade-off between existing and new classes. Our results show that simple geometric regularization of class representations offers an effective tool for continual learning.

Original languageEnglish
Title of host publicationInternational Conference on Learning Representations 2022
EditorsYann LeCun
Place of PublicationUSA
PublisherOpenReview
Number of pages16
Publication statusPublished - 2022
Externally publishedYes
EventInternational Conference on Learning Representations 2022 - Online, United States of America
Duration: 25 Apr 202229 Apr 2022
Conference number: 10th
https://openreview.net/group?id=ICLR.cc/2022/Conference (Peer Reviews)
https://iclr.cc/Conferences/2022 (Website)

Conference

ConferenceInternational Conference on Learning Representations 2022
Abbreviated titleICLR 2022
Country/TerritoryUnited States of America
Period25/04/2229/04/22
Internet address

Cite this