We present a novel approach for extending the input space around unmodified mobile devices. Using built-in front-facing cameras of unmodified handheld devices, GlassHands estimates hand poses and gestures through reflections in sunglasses, ski goggles or visors. Thereby, GlassHands creates an enlarged input space, rivaling input reach on large touch displays. We introduce the idea along with its technical concept and implementation. We demonstrate the feasibility and potential of our proposed approach in several application scenarios, such as map browsing or drawing using a set of interaction techniques previously possible only with modified mobile devices or on large touch displays. Our research is backed up with a user study.
In consecutive research we discuss, how to extend this beyond the use of reflective glasses.
Media
30 Second Teaser
Main Video (4:22 min)
Publications
Jens Grubert, Eyal Ofek, Michel Pahud, Matthias Kranz and Dieter Schmalstieg. GlassHands: Interaction Around Unmodified Mobile Devices Using Sunglasses. In Proceedings of the 2016 ACM Conference on Interactive Surfaces and Spaces (pp. 215-224).
preprint | doi
Jens Grubert, Eyal Ofek, Michel Pahud, Matthias Kranz and Dieter Schmalstieg. Towards Interaction Around Unmodified Camera-equipped Mobile Devices. In Cross-Surface Workshop in Conjunction with the 2016 ACMConference on Interactive Surfaces and Spaces.
preprint | workshop website
