Ibsen, Mathias, Nichols, Robert, Rathgeb, Christian, Robertson, David J., Davis, Josh P. ORCID: https://orcid.org/0000-0003-0017-7159, Løvåsdal, Frøy, Raja, Kiran, Jenkins, Ryan E. and Busch, Christoph (2024) Conditional face image manipulation detection: combining algorithm and human examiner decisions. In: IH&MMSec '24: Proceedings of the 2024 ACM Workshop on Information Hiding and Multimedia Security. Association for Computing Machinery (ACM), New York, USA, pp. 41-46. ISBN 979-8400706370 (doi:10.1145/3658664.3659649)

	PDF (Conference Proceedings) 48178 DAVIS_Conditional_Face_Image_Manipulation_Detection_(OA PROCEEDINGS)_2024.pdf - Published Version Restricted to Repository staff only Download (119MB) | Request a copy
Preview	PDF (Open Access Conference Paper) 48178 DAVIS_Conditional_Face_Image_Manipulation_Detection_(OA PAPER)_2024.pdf - Published Version Available under License Creative Commons Attribution. Download (10MB) | Preview

Abstract

It has been shown that digitally manipulated face images can pose a security threat to automated authentication systems (e.g., when such systems are used for border control). In such scenarios, a malicious actor can, in many countries, apply for an identity document using a manipulated face image, which can then be used to gain fraudulent access to a system. Research has shown that humans and algorithms struggle to detect digitally manipulated face images, especially when the type of manipulation is unknown or when evaluated across multiple types of manipulations. In this work, we consider the detection performance of algorithms and humans on datasets consisting of retouched, face swapped and morphed images. Specifically, we investigate the joint performance of algorithms and humans in a differential detection scenario where both a trusted and suspected image are presented simultaneously. To this end, we propose a conditional face image manipulation detection approach where the human decision is only considered when the algorithm is unsure about the decision outcome. The results show that the automated algorithm performs better than the human detectors and that combining the decisions of algorithms and humans, in general, leads to an increased detection performance. To our knowledge, this is the first study to explore the joint detection performance of algorithms and humans in a differential face manipulation detection scenario and when using a variety of face image manipulations.

Item Type:	Conference Proceedings
Title of Proceedings:	IH&MMSec '24: Proceedings of the 2024 ACM Workshop on Information Hiding and Multimedia Security
Additional Information:	The ACM Special Interest Group on Multimedia provides a forum for researchers, engineers, and practitioners in all aspects of multimedia computing, communication, storage, and applications. SIGMM sponsors the ACM Multimedia Conference series and ad hoc workshops on emerging areas of multimedia. In addition, SIGMM supports the upcoming "ACM Transactions on Multimedia, Applications, and Computing" (TOMCCAP, early 2005) and the SIGMM Website which contains forums and other relevant material. All SIGMM publications are available through the ACM Digital Library. SIGMM members receive a copy of the ACM Multimedia Conference proceedings on CD and a significant discount on registration fees for SIGMM sponsored events.
Uncontrolled Keywords:	face recognition, morphs
Subjects:	B Philosophy. Psychology. Religion > BF Psychology Q Science > QA Mathematics > QA75 Electronic computers. Computer science T Technology > T Technology (General)
Faculty / School / Research Centre / Research Group:	Faculty of Education, Health & Human Sciences Faculty of Education, Health & Human Sciences > Institute for Lifecourse Development Faculty of Education, Health & Human Sciences > Institute for Lifecourse Development > Centre for Thinking and Learning Faculty of Education, Health & Human Sciences > School of Human Sciences (HUM)
Last Modified:	09 Jan 2026 12:35
URI:	https://gala.gre.ac.uk/id/eprint/48178

Actions (login required)

View Item

Downloads