Remote Monitoring and Teleoperation of Autonomous Vehicles — Is Virtual Reality an Option?

Abstract

While the promise of autonomous vehicles has led to significant scientific and industrial progress, fully automated, SAE level 5 conform cars will likely not see mass adoption anytime soon. Instead, in many applications, human supervision, such as remote monitoring and teleoperation, will be required for the foreseeable future. While Virtual Reality (VR) has been proposed as one potential interface for teleoperation, its benefits and drawbacks over physical monitoring and teleoperation solutions have not been thoroughly investigated. To this end, we contribute three user studies, comparing and quantifying the performance of and subjective feedback for a VR-based system with an existing monitoring and teleoperation system, which is in industrial use today. Through these three user studies, we contribute to a better understanding of future virtual monitoring and teleoperation solutions for autonomous vehicles. The results of our first user study (n=16) indicate that a VR interface replicating the physical interface does not outperform the physical interface. It also quantifies the negative effects that combined monitoring and teleoperating tasks have on users irrespective of the interface being used. The results of the second user study (n=24) indicate that the perceptual and ergonomic issues caused by VR outweigh its benefits, like better concentration through isolation. The third follow-up user study (n=24) specifically targeted the perceptual and ergonomic issues of VR; the subjective feedback of this study indicates that newer-generation VR headsets have the potential to catch up with the current physical displays.

Media

Conditions of Study 1, from left to right: The physical BASELINE interface used in industry in the MONITORING scenario. The replicated VR interface in the MONITORING scenario. The physical BASELINE interface in the TELEOPERATION scenario. The replicated VR interface in the TELEOPERATION scenario.

Video

Publication

Snehanjali Kalamkar, Verena Biener, Fabian Beck & Jens Grubert. Remote Monitoring and Teleoperation of Autonomous Vehicles — Is Virtual Reality an Option? In Proceedings of IEEE International Symposium on Mixed and Augmented Reality 2023 (ISMAR 2023), pp. 463-472. arxiv preprint | DOI | Daten