Around-Device Interaction using Corneal Imaging

Abstract

Around-device interaction techniques aim at extending the input space using various sensing modalities on mobile and wearable devices. We aim at extending the input area of mobile devices using front-facing device-centered cameras that capture reflections in the human eye. As current generation mobile devices lack high resolution front-facing cameras we study the feasibility of around device interaction using corneal reflective imaging based on a high resolution camera. We present a workflow, a technical prototype and an evaluation, including a migration path from high resolution to low resolution imagers. Our study indicates, that under optimal conditions a spatial sensing resolution of 5 cm in the vicinity of a mobile phone is possible.

Media

Pipeline Results
Results of single steps of the pipeline, (a) original camera picture, (b) eye region, (c) eye region with limbus, (d) Unwrapping result, (e) Object detection (red and blue rectangle) (f) distance of objects.
Evaluation Setup
Setup of the laboratory study. Left: Condition with red rectangular and blue marker. Right: Condition with smartphone and finger.
Results for Scenario 1
Errors (x, y, RMS) across all resolutions for condition with red rectangular and blue marker.

Publications

  • Daniel Schneider & Jens Grubert. Towards Around-Device Interaction using Corneal Imaging. In Proceedings of the 2017 ACM Conference on Interactive Surfaces and Spaces (ISS 2017), Brighton, UK, pp. 287–293. arxiv preprint | DOI
  • Daniel Schneider & Jens Grubert. Feasibility of Corneal Imaging for Handheld Augmented Reality. n Adjunct Proceedings of the 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2017), Nantes, France, pp. 44-45. arxiv preprint
    • | DOI