Nwokeji, Joshua, Nkwo, Makuochi S. ORCID: https://orcid.org/0000-0002-9774-9602, Ikwunne, Tochukwu and Yeerbo, Meiyeer (2026) UX experts vs. AI: exploring the performance of large language models and humans on detecting dark patterns. AI and Ethics, 6:288. ISSN 2730-5961 (Online) (doi:10.1007/s43681-026-01151-x)

Preview

PDF (Open Access Article) 54416 NKWO_UX_Experts_Vs_AI_(OA)_2026.pdf - Published Version Available under License Creative Commons Attribution. Download (2MB) | Preview

Abstract

While AI ethics ensures fairness, accountability, and protection of user rights, dark patterns manipulate users to take unintended actions on digital interfaces. Related studies uncover limited insights into how reliably; human experts and AI models can detect dark patterns within a specific taxonomy. Our research fills this gap by asymmetrically examining cross-origin detection performance of human and AI/LLM evaluators (each evaluator’s ability to detect dark patterns generated by the opposite source) to understand their limitations and future potentials. Using GPT-4.1, we generated 200 UI images (with matched dark and non-dark pattern pairs) and selected 200 UI images collected 200 human-created UI screenshots from the ContextDP/AidUI dataset, based on computational, methodological, and statistical considerations. We calculated inter-rater reliability, recall, and error distribution. The results show that UX experts achieved substantial agreement (k = 0.75) and significantly higher recall (r = 0.99) over AI/LLMs. We present a novel study which explore the performance of AI/LLMs and UX experts in detecting dark patterns in UI images, and provide a benchmark dataset that could be useful to future research, while discussing empirical insights into the role, limitations, and promise of AI/LLMs in UI/UX design ethics and auditing, in realistic deployment scenarios.

Item Type:	Article
Uncontrolled Keywords:	dark patterns, deceptive design, human-centered AI, AI ethics, large language models, quantitative study
Subjects:	Q Science > Q Science (General) Q Science > QA Mathematics Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Faculty / School / Research Centre / Research Group:	Faculty of Engineering & Science Faculty of Engineering & Science > School of Computing & Mathematical Sciences (CMS)
Last Modified:	18 Sep 2026 09:31
URI:	https://gala.gre.ac.uk/id/eprint/54416

Actions (login required)

View Item

Downloads