VocabulARy: Learning Vocabulary in AR Supported by Keyword Visualisations

Abstract

Learning vocabulary in a primary or secondary language is enhanced when we encounter words in context. This context can be afforded by the place or activity we are engaged with. Existing learning environments include formal learning, mnemonics, flashcards, use of a dictionary or thesaurus, all leading to practice with new words in context. In this work, we propose an enhancement to the language learning process by providing the user with words and learning tools in context, with VocabulARy. VocabulARy visually annotates objects in AR, in the user's surroundings, with the corresponding English (first language) and Japanese (second language) words to enhance the language learning process. In addition to the written and audio description of each word, we also present the user with a keyword and its visualisation to enhance memory retention. We evaluate our prototype by comparing it to an alternate AR system that does not show an additional visualisation of the keyword, and, also, we compare it to two non-AR systems on a tablet, one with and one without visualising the keyword. Our results indicate that AR outperforms the tablet system regarding immediate recall, mental effort and task completion time. Additionally, the visualisation approach scored significantly higher than showing only the written keyword with respect to immediate and delayed recall and learning efficiency, mental effort and task-completion time.

Media

Vocabulary Teaser
a) Participant interacting with VocabulARy during the study. b) VocabulARy prototype though HoloLens2 in KEYWORD + VISUALISATION instruction mode (the Japanese word “hagaki” sounds as the English phrase “hug a key” (keyword) and visualised with an animated hand grabbing a key (visualisation)). c) Participant interacting with non-AR version of VocabulARy in KEYWORD instruction mode (Note there is no visualisation of the keyword). AR and non-AR condition were tested with both instruction modes..

Videos

Teaser

Publication

Maheshya Weerasinghe, Verena Biener, Jens Grubert, Aaron J. Quigley, Alice Toniolo, Klen Čopič Pucihar & Matjaž Kljun. VocabulARy: Learning Vocabulary in AR Supported by Keyword Visualisations. In IEEE Transactions on Visualization and Computer Graphics (TVCG), vol. 28, no. 11, Nov. 2022, pp. 3748-3758. arxiv preprint | DOI

Replication

Maheshya Weerasinghe, Verena Biener, Jens Grubert, Aaron J. Quigley, Alice Toniolo, Klen Čopič Pucihar & Matjaž Kljun. VocabulARy replicated: comparing teenagers to young adults. In Adjunct Proceedings of IEEE International Symposium on Mixed and Augmented Reality 2022 (ISMAR), pp. 273-285. arxiv preprint