Metatla, Oussama, Correia, Nuno N. ORCID: https://orcid.org/0000-0002-9999-9162, Martin, Fiore, Bryan-Kinns, Nick and Stockman, Tony (2016) Tap the shapeTones: Exploring the effects of crossmodal congruence in an audio-visual interface. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. CHI '16 . ACM, New York, NY, USA, pp. 1055-1066. ISBN 978-1-4503-3362-7 (doi:10.1145/2858036.2858456)

Preview

PDF (Author Accepted Manuscript) 20968 CORREIA_Tap_the_ShapeTones_2016.pdf - Accepted Version Download (1MB) | Preview

Abstract

There is growing interest in the application of crossmodal perception to interface design. However, most research has focused on task performance measures and often ignored user experience and engagement. We present an examination of crossmodal congruence in terms of performance and engagement in the context of a memory task of audio, visual, and audio-visual stimuli. Participants in a first study showed improved performance when using a visual congruent mapping that was cancelled by the addition of audio to the baseline conditions, and a subjective preference for the audio-visual stimulus that was not reflected in the objective data. Based on these findings, we designed an audio-visual memory game to examine the effects of crossmodal congruence on user experience and engagement. Results showed higher engagement levels with congruent displays with some reported preference for potential challenge and enjoyment that an incongruent display may support, particularly for increased task complexity.

Item Type:	Conference Proceedings
Title of Proceedings:	Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems
Additional Information:	Conference held from 07 - 12 May 2016, San Jose, California, USA.
Uncontrolled Keywords:	Games, user experience, User engagement, audio-visual display, crossmodal congruence, spatial mappings
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Faculty / School / Research Centre / Research Group:	Faculty of Engineering & Science > School of Computing & Mathematical Sciences (CMS) Faculty of Engineering & Science
Last Modified:	04 Mar 2022 13:07
URI:	http://gala.gre.ac.uk/id/eprint/20968

Actions (login required)

View Item

Downloads