Activity recognition and abnormality detection with the switching hidden semi-Markov model

Thi V. Duong, Hung H. Bui, Dinh Q. Phung, Svetha Venkatesh

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

435 Citations (Scopus)


This paper addresses the problem of learning and recognizing human activities of daily living (ADL), which is an important research issue in building a pervasive and smart environment. In dealing with ADL, we argue that it is beneficial to exploit both the inherent hierarchical organization of the activities and their typical duration. To this end, we introduce the Switching Hidden Semi-Markov Model (S-HSMM), a two-layered extension of the hidden semi-Markov model (HSMM) for the modeling task. Activities are modeled in the S-HSMM in two ways: the bottom layer represents atomic activities and their duration using HSMMs; the top layer represents a sequence of high-level activities where each high-level activity is made of a sequence of atomic activities. We consider two methods for modeling duration: the classic explicit duration model using multinomial distribution, and the novel use of the discrete Coxian distribution. In addition, we propose an effective scheme to detect abnormality without the need for training on abnormal data. Experimental results show that the S-HSMM performs better than existing models including the flat HSMM and the hierarchical hidden Markov model in both classification and abnormality detection tasks, alleviating the need for presegmented training data. Furthermore, our discrete Coxian duration model yields better computation time and generalization error than the classic explicit duration model

Original languageEnglish
Title of host publicationProceedings - 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages8
ISBN (Print)0769523722, 9780769523729
Publication statusPublished - 1 Jan 2005
Externally publishedYes
EventIEEE Conference on Computer Vision and Pattern Recognition 2005 - San Diego, United States of America
Duration: 20 Jun 200525 Jun 2005 (Proceedings)

Publication series

NameProceedings - 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005


ConferenceIEEE Conference on Computer Vision and Pattern Recognition 2005
Abbreviated titleCVPR 2005
Country/TerritoryUnited States of America
CitySan Diego
Internet address

Cite this