Aranyi, Gabor, Pecune, Florian, Charles, Fred, Pelachaud, Catherine and Cavazza, Marc ORCID: 0000-0001-6113-9696 (2016) Affective interaction with a virtual character through an fNIRS brain-computer interface. Frontiers in Computational Neuroscience, 10:70. ISSN 1662-5188 (Online) (doi:https://doi.org/10.3389/fncom.2016.00070)

Preview

PDF (Publisher's PDF - Open Access) 19802 CAVAZZA_Affective_Interaction_with_a_Virtual_Character_(OA)_2016.pdf - Published Version Available under License Creative Commons Attribution. Download (1MB) | Preview

Abstract

Affective brain-computer interfaces (BCI) harness Neuroscience knowledge to develop affective interaction from first principles. In this article, we explore affective engagement with a virtual agent through Neurofeedback (NF). We report an experiment where subjects engage with a virtual agent by expressing positive attitudes towards her under a NF paradigm. We use for affective input the asymmetric activity in the dorsolateral prefrontal cortex (DL-PFC), which has been previously found to be related to the high-level affective-motivational dimension of approach/avoidance. The magnitude of left-asymmetric DL-PFC activity, measured using functional near infrared spectroscopy (fNIRS) and treated as a proxy for approach, is mapped onto a control mechanism for the virtual agent’s facial expressions, in which action units (AUs) are activated through a neural network. We carried out an experiment with 18 subjects, which demonstrated that subjects are able to successfully engage with the virtual agent by controlling their mental disposition through NF, and that they perceived the agent’s responses as realistic and consistent with their projected mental disposition. This interaction paradigm is particularly relevant in the case of affective BCI as it facilitates the volitional activation of specific areas normally not under conscious control. Overall, our contribution reconciles a model of affect derived from brain metabolic data with an ecologically valid, yet computationally controllable, virtual affective communication environment.

Item Type:	Article
Additional Information:	Copyright © 2016 Aranyi, Pecune, Charles, Pelachaud and Cavazza. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Uncontrolled Keywords:	brain-computer interfaces, fNIRS, neurofeedback, affective computing, virtual agents
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Faculty / School / Research Centre / Research Group:	Faculty of Engineering & Science > School of Computing & Mathematical Sciences (CMS) Faculty of Engineering & Science
Last Modified:	04 Mar 2022 13:07
URI:	http://gala.gre.ac.uk/id/eprint/19802

Actions (login required)

View Item

Downloads