Digits: Freehand 3D interactions anywhere using a wrist-worn gloveless sensor

David Kim, Otmar Hilliges, Shahram Izadi, Alex Butler, Jiawen Chen, Iason Oikonomidis, Patrick Olivier

    Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

    297 Citations (Scopus)

    Abstract

    Digits is a wrist-worn sensor that recovers the full 3D pose of the user's hand. This enables a variety of freehand interactions on the move. The system targets mobile settings, and is specifically designed to be low-power and easily reproducible using only off-the-shelf hardware. The electronics are self-contained on the user's wrist, but optically image the entirety of the user's hand. This data is processed using a new pipeline that robustly samples key parts of the hand, such as the tips and lower regions of each finger. These sparse samples are fed into new kinematic models that leverage the biomechanical constraints of the hand to recover the 3D pose of the user's hand. The proposed system works without the need for full instrumentation of the hand (for example using data gloves), additional sensors in the environment, or depth cameras which are currently prohibitive for mobile scenarios due to power and form-factor considerations. We demonstrate the utility of Digits for a variety of application scenarios, including 3D spatial interaction with mobile devices, eyes-free interaction on-the-move, and gaming. We conclude with a quantitative and qualitative evaluation of our system, and discussion of strengths, limitations and future work.

    Original languageEnglish
    Title of host publicationUIST'12 - Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology
    Pages167-176
    Number of pages10
    Publication statusPublished - 19 Nov 2012
    EventACM Symposium on User Interface Software and Technology 2012 - Cambridge, United States of America
    Duration: 7 Oct 201210 Oct 2012
    Conference number: 25th
    https://dl.acm.org/doi/proceedings/10.1145/2380116

    Conference

    ConferenceACM Symposium on User Interface Software and Technology 2012
    Abbreviated titleUIST 2012
    CountryUnited States of America
    CityCambridge
    Period7/10/1210/10/12
    Internet address

    Keywords

    • 3D interaction
    • Hand tracking
    • Mobile
    • Wearables

    Cite this