Replication Papers

The "replication crisis" describes widespread failures to replicate empirical findings. Over 50% of previously reported results could only be replicated in a few cases. While originating in medicine and psychology, the replication crisis has also reached other empirical research areas, such as computer science. These issues have shaken the foundations of these fields: Can empirical results be believed? How can we support the reproducibility of increasingly complex interactive systems? Here, we present our publications that replicated prior user studies and explored the similarities and differences in the results. These publications were submitted to the series "Workshop on Replication in Extended Reality (WoRXR)", held at IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

Media

Replicated conditions across user studies
We conducted 2 studies with partially replicated conditions. In study 1 (top row), we compared a baseline interface for monitoring and teleoperation of remote automated vehicles (a) with a version optimized for monitoring only on a physical monitor (b) as well as in VR (c). In study 2 (bottom row), we replicated two out of these three conditions (d, e) and added another VR condition.

Publications

  • Snehanjali Kalamkar, Verena Biener & Jens Grubert. VR for Remote Monitoring of Automated Vehicles: Replications Across Follow-Up Evaluations. In Adjunct Proceedings of IEEE International Symposium on Mixed and Augmented Reality 2023 (ISMAR 2023), pp. 3-4. DOI
  • Maheshya Weerasinghe, Verena Biener, Jens Grubert, Aaron J. Quigley, Alice Toniolo, Klen Čopič Pucihar & Matjaž Kljun. VocabulARy replicated: comparing teenagers to young adults. In Adjunct Proceedings of IEEE International Symposium on Mixed and Augmented Reality 2022 (ISMAR), pp. 273-285. arxiv preprint
  • Verena Biener & Jens Grubert. Content Transfer Across Multiple Screens with Combined Eye-Gaze and Touch Interaction - A Replication Study. In Adjunct Proceedings of IEEE International Symposium on Mixed and Augmented Reality 2022 (ISMAR 2022), pp.270-273 . arxiv preprint