Multi-scale conditional random fields for first-person activity recognition

Kai Zhan, Steven Faux, Fabio Ramos

Research output: Chapter in Book/Report/Conference proceedingConference PaperOtherpeer-review

35 Citations (Scopus)


We propose a novel pervasive system to recognise human daily activities from a wearable device. The system is designed in a form of reading glasses, named 'Smart Glasses', integrating a 3-axis accelerometer and a first-person view camera. Our aim is to classify user's activities of daily living (ADLs) based on both vision and head motion data. This ego-activity recognition system not only allows caretakers to track on a specific person (such as patient or elderly people), but also has the potential to remind/warn people with cognitive impairments of hazardous situations. We present the following contributions in this paper: a feature extraction method from accelerometer and video; a classification algorithm integrating both locomotive (body motions) and stationary activities (without or with small motions); a novel multi-scale dynamic graphical model structure for structured classification over time. We collect, train and validate our system on a large dataset containing 20 hours of ADLs data, including 12 daily activities under different environmental settings. Our method improves the classification performance (F-Score) of conventional approaches from 43.32%(video features) and 66.02%(acceleration features) by an average of 20-40% to 84.45%, with an overall accuracy of 90.04% in realistic ADLs.

Original languageEnglish
Title of host publication2014 IEEE International Conference on Pervasive Computing and Communications, PerCom 2014
PublisherIEEE Computer Society
Number of pages9
Publication statusPublished - 2014
Externally publishedYes
EventIEEE International Conference on Pervasive Computing and Communications 2014 - Budapest, Hungary
Duration: 24 Mar 201428 Mar 2014
Conference number: 12th (Proceedings)


ConferenceIEEE International Conference on Pervasive Computing and Communications 2014
Abbreviated titlePerCom 2014
Internet address

Cite this