Wang, Rui, Huang, Huaibo, Zhang, Xufeng, Ma, Jixin and Zheng, Aihua (2019) A novel distance learning for elastic cross-modal audio-visual matching. In: 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), Shanghai, China, 8th - 12th July 2019. IEEE Xplore . Institute of Electrical and Electronics Engineers (IEEE), Piscataway, New Jersey, pp. 300-305. ISBN 978-1538692141; 978-1538692134; 978-1538692158 (doi:10.1109/ICMEW.2019.00-70)

Preview

PDF (Abstract of paper) 36277_MA_A_novel_distance_learning_for_elastic_cross_modal_audio_visual_matching.pdf - Other Download (67kB) | Preview

Abstract

In this work we propose a novel network formulation for joint representation of cross-modal audio and visual information base on metric learning. We employ a distance learning framework as a training procedure. For this purpose we introduce an elastic matching network (EmNet) and a novel loss function to learn the shared latent space representation of multi-modal information. The elastic matching network is capable of matching given face image (or audio voice clip) from diverse number of audio clips (or face images). We quantitatively and qualitatively evaluate the purposed approach on the standard audio-visual matching evaluation dataset, the overlap of VoxCeleb and VGGFace by both multi-way and binary audio-visual matching tasks. The promising performance comparing to the existing methods verifies the effectiveness of the proposed approach, which yields to a new state-of-the-art for cross-modal audio-visual matching.

Item Type:	Conference Proceedings
Title of Proceedings:	2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), Shanghai, China, 8th - 12th July 2019
Uncontrolled Keywords:	distance learning; elastic cross-modal audio-visual matching; cross-modality; audio-visual matching; elastic multi-way matching; distance learning
Subjects:	H Social Sciences > H Social Sciences (General) L Education > L Education (General) T Technology > T Technology (General)
Faculty / School / Research Centre / Research Group:	Faculty of Engineering & Science Faculty of Engineering & Science > School of Computing & Mathematical Sciences (CMS)
Last Modified:	19 Oct 2023 12:43
URI:	http://gala.gre.ac.uk/id/eprint/36277

Actions (login required)

View Item

Downloads