MultiFi: Interaction with Body-Proximate Display Ecologies


Display devices on and around the body such as smartwatches, head-mounted displays or tablets enable users to interact on the go. However, diverging input and output fidelities of these devices can lead to interaction seams that can inhibit effcient mobile interaction, when users employ multiple devices at once. We present MultiFi, an interactive system that combines the strengths of multiple displays and overcomes the seams of mobile interaction with widgets distributed over multiple devices. A comparative user study indicates that combined head-mounted display and smartwatch interfaces can outperform interaction with single wearable devices. In consecutive research we have identified major challenges for body proximate display ecosystems encompassing design, technology, perceptual and social issues.


30 Second Teaser

Main Video (3:30 min)

Supplemental Material

Extended study description including details of statistical tests.


  • Jens Grubert, Matthias Heinisch, Aaron Quigley and Dieter Schmalstieg. MultiFi: Multi-Fidelity Interaction with Displays On and Around the Body. In Proceedings of ACM CHI 2015, April 18-23 2015, Seoul (pp. 3933-3942). preprint | presentation | teaser video | full videoextended study description | video of study conditions
  • Jens Grubert, Matthias Kranz and Aaron Quigley. Design and Technology Challenges for Body Proximate Display Ecosystems. In MobileHCI15 Adjunct, Workshop From Mobile to Wearable: Using Wearable Devices to Enrich Mobile Interaction, pp. 951-954. pdf | workshop website | doi
  • Aaron Quigley and Jens Grubert. Perceptual and Social Challenges in Body Proximate Display Ecosystems. In MobileHCI15 Adjunct, Workshop Mobile Collocated Interactions With Wearables, pp. 1168-1174. pdf | workshop website | doi
  • Jens Grubert, Matthias Kranz and Aaron Quigley. Challenges in Mobile Multi-Device Ecosystems. In mUX: The Journal of Mobile User Experience, 5(1), 2016, pp. 1-22. open access versionarxiv preprint | doi