Skip navigation

Tap the shapeTones: Exploring the effects of crossmodal congruence in an audio-visual interface

Tap the shapeTones: Exploring the effects of crossmodal congruence in an audio-visual interface

Metatla, Oussama, Correia, Nuno N. ORCID: 0000-0002-9999-9162, Martin, Fiore, Bryan-Kinns, Nick and Stockman, Tony (2016) Tap the shapeTones: Exploring the effects of crossmodal congruence in an audio-visual interface. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. CHI '16 . ACM, New York, NY, USA, pp. 1055-1066. ISBN 978-1-4503-3362-7 (doi:https://doi.org/10.1145/2858036.2858456)

[img]
Preview
PDF (Author Accepted Manuscript)
20968 CORREIA_Tap_the_ShapeTones_2016.pdf - Accepted Version

Download (1MB) | Preview

Abstract

There is growing interest in the application of crossmodal perception to interface design. However, most research has focused on task performance measures and often ignored user experience and engagement. We present an examination of crossmodal congruence in terms of performance and engagement in the context of a memory task of audio, visual, and audio-visual stimuli. Participants in a first study showed improved performance when using a visual congruent mapping that was cancelled by the addition of audio to the baseline conditions, and a subjective preference for the audio-visual stimulus that was not reflected in the objective data. Based on these findings, we designed an audio-visual memory game to examine the effects of crossmodal congruence on user experience and engagement. Results showed higher engagement levels with congruent displays with some reported preference for potential challenge and enjoyment that an incongruent display may support, particularly for increased task complexity.

Item Type: Conference Proceedings
Title of Proceedings: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems
Additional Information: Conference held from 07 - 12 May 2016, San Jose, California, USA.
Uncontrolled Keywords: Games, user experience, User engagement, audio-visual display, crossmodal congruence, spatial mappings
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Faculty / Department / Research Group: Faculty of Architecture, Computing & Humanities
Faculty of Architecture, Computing & Humanities > Department of Computing & Information Systems
Last Modified: 16 Aug 2019 09:38
Selected for GREAT 2016: None
Selected for GREAT 2017: None
Selected for GREAT 2018: None
Selected for GREAT 2019: GREAT 2
URI: http://gala.gre.ac.uk/id/eprint/20968

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year

View more statistics