The eyes know it: FakeET- an eye-tracking database to understand deepfake perception

Parul Gupta, Komal Chugh, Abhinav Dhall, Ramanathan Subramanian

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

We present FakeET - an eye-tracking database to understand human visual perception of deepfake videos. Given that the principal purpose of deepfakes is to deceive human observers, FakeET is designed to understand and evaluate the ability of viewers to detect synthetic video artifacts. FakeET contains viewing patterns compiled from 40 users via the Tobii desktop eye-tracker for 811 videos from the Google Deepfake dataset, with a minimum of two viewings per video. Additionally, EEG responses acquired via the Emotiv sensor are also available. The compiled data confirms (a) distinct eye movement characteristics for real vs fake videos; (b) utility of the eye-track saliency maps for spatial forgery localization and detection, and (c) Error Related Negativity (ERN) triggers in the EEG responses, and the ability of the raw EEG signal to distinguish between real and fake videos.

Original languageEnglish
Title of host publicationProceedings of the 2020 International Conference on Multimodal Interaction
EditorsNadia Berthouze, Mohamed Chetouani, Mikio Nakano
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Pages519-527
Number of pages9
ISBN (Electronic)9781450375818
DOIs
Publication statusPublished - 2020
EventInternational Conference on Multimodal Interaction 2020 - Virtual , Netherlands
Duration: 25 Oct 202029 Oct 2020
Conference number: 22nd
https://icmi.acm.org/2020/index.php?id=cfp (Website)
https://dl.acm.org/doi/proceedings/10.1145/3382507 (Proceedings)

Conference

ConferenceInternational Conference on Multimodal Interaction 2020
Abbreviated titleICMI 2020
CountryNetherlands
CityVirtual
Period25/10/2029/10/20
Internet address

Keywords

  • deepfake
  • eeg
  • eye-tracking
  • visual perception

Cite this