Deep auto-set: a deep auto-encoder-set network for activity recognition using wearables

Alireza Abedin Varamin, Ehsan Abbasnejad, Qinfeng Shi, Damith C. Ranasinghe, Hamid Rezatofighi

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

21 Citations (Scopus)


Automatic recognition of human activities from time-series sensor data (referred to as HAR) is a growing area of research in ubiquitous computing. Most recent research in the field adopts supervised deep learning paradigms to automate extraction of intrinsic features from raw signal inputs and addresses HAR as a multi-class classification problem where detecting a single activity class within the duration of a sensory data segment suffices. However, due to the innate diversity of human activities and their corresponding duration, no data segment is guaranteed to contain sensor recordings of a single activity type. In this paper, we express HAR more naturally as a set prediction problem where the predictions are sets of ongoing activity elements with unfixed and unknown cardinality. For the first time, we address this problem by presenting a novel HAR approach that learns to output activity sets using deep neural networks. Moreover, motivated by the limited availability of annotated HAR datasets as well as the unfortunate immaturity of existing unsupervised systems, we complement our supervised set learning scheme with a prior unsupervised feature learning process that adopts convolutional auto-encoders to exploit unlabeled data. The empirical experiments on two widely adopted HAR datasets demonstrate the substantial improvement of our proposed methodology over the baseline models.

Original languageEnglish
Title of host publicationProceedings of the 15th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services
EditorsCristian Borcea, Shiwen Mao, Jian Tang
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Number of pages8
ISBN (Electronic)9781450360937
Publication statusPublished - Nov 2018
Externally publishedYes
EventInternational Conference on Mobile and Ubiquitous Systems: Networks and Services 2018 - New York, United States of America
Duration: 5 Nov 20187 Nov 2018
Conference number: 15th (Proceedings) (Website)


ConferenceInternational Conference on Mobile and Ubiquitous Systems: Networks and Services 2018
Abbreviated titleMobiQuitous 2018
Country/TerritoryUnited States of America
CityNew York
Internet address


  • Activity recognition
  • Deep learning
  • Time-series data
  • Wearable sensors

Cite this