Multi-scale conditional random fields for first-person activity recognition on elders and disabled patients

Kai Zhan, Steven Faux, Fabio Ramos

Research output: Contribution to journalArticleResearchpeer-review

32 Citations (Scopus)


We propose a novel pervasive system to recognise human daily activities from a wearable device. The system is designed in a form of reading glasses, named 'Smart Glasses', integrating a 3-axis accelerometer and a first-person view camera. Our aim is to classify subject's activities of daily living (ADLs) based on their vision and head motion data. This ego-activity recognition system not only allows caretakers to track on a specific person (such as disabled patient or elderly people), but also has the potential to remind/warn people with cognitive impairments of hazardous situations. We present the following contributions: a feature extraction method from accelerometer and video; a classification algorithm integrating both locomotive (body motions) and stationary activities (without or with small motions); a novel multi-scale dynamic graphical model for structured classification over time. In this paper, we collect, train and validate our system on two large datasets: 20 h of elder ADLs datasets and 40 h of patient ADLs datasets, containing 12 and 14 different activities separately. The results show that our method efficiently improves the system performance (F-Measure) over conventional classification approaches by an average of 20%-40% up to 84.45%, with an overall accuracy of 90.04% for elders. Furthermore, we also validate our method on 30 patients with different disabilities, achieving an overall accuracy up to 77.07%.
Original languageEnglish
Pages (from-to)251-267
Number of pages17
JournalPervasive and Mobile Computing
Publication statusPublished - Jan 2015
Externally publishedYes


  • Activity recognition
  • Feature classification
  • Graphical model
  • First-person
  • Feature extraction
  • Computer vision

Cite this