A multi-party multi-modal dataset for focus of visual attention in human-human and human-robot interaction

Kalin Stefanov, Jonas Beskow

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

12 Citations (Scopus)


This papers describes a data collection setup and a newly recorded dataset. The main purpose of this dataset is to explore patterns in the focus of visual attention of humans under three different conditions - two humans involved in task-based interaction with a robot; same two humans involved in task-based interaction where the robot is replaced by a third human, and a free three-party human interaction. The dataset contains two parts - 6 sessions with duration of approximately 3 hours and 9 sessions with duration of approximately 4.5 hours. Both parts of the dataset are rich in modalities and recorded data streams - they include the streams of three Kinect v2 devices (color, depth, infrared, body and face data), three high quality audio streams, three high resolution GoPro video streams, touch data for the task-based interactions and the system state of the robot. In addition, the second part of the dataset introduces the data streams from three Tobii Pro Glasses 2 eye trackers. The language of all interactions is English and all data streams are spatially and temporally aligned.

Original languageEnglish
Title of host publicationProceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)
EditorsNicoletta Calzolari, Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Place of PublicationStroudsburg PA USA
PublisherEuropean Language Resources Association (ELRA)
Number of pages5
ISBN (Electronic)9782951740891
Publication statusPublished - 2016
Externally publishedYes
EventInternational Conference on Language Resources and Evaluation 2016 - Portoroz, Slovenia
Duration: 23 May 201628 May 2016
Conference number: 10th
http://www.lrec-conf.org/proceedings/lrec2016/index.html (Proceedings)


ConferenceInternational Conference on Language Resources and Evaluation 2016
Abbreviated titleLREC 2016
Internet address


  • Dataset
  • Focus of visual attention
  • Human-human interaction
  • Human-robot interaction

Cite this