Uher, Jana ORCID: https://orcid.org/0000-0003-2450-4943 (2025) Statistics is not measurement: the inbuilt semantics of psychometric scales and language-based models obscures crucial epistemic differences. Frontiers in Psychology, 16:1534270. ISSN 1664-1078 (Online) (doi:10.3389/fpsyg.2025.1534270)

Preview

PDF (Open Access Article) 50862 UHER_Statistics_Is_Not_Measurement_The_Inbuilt_Semantics_Of_Psychometric_Scales_(OA)_2025.pdf - Published Version Available under License Creative Commons Attribution. Download (11MB) | Preview

Abstract

This article provides a comprehensive critique of psychology's overreliance on statistical modelling at the expense of epistemologically grounded measurement processes. It highlights that statistics deals with structural relations in data regardless of what these data represent, whereas measurement establishes traceable empirical relations between the phenomena studied and the data representing information about them. These crucial epistemic differences are elaborated using Rosen's general model of measurement, involving the coherent modelling of the (1) objects of research, (2) data generation (encoding), (3) formal manipulation (e.g., statistical analysis) and (4) result interpretation regarding the objects studied (decoding). This system of interrelated modelling relations is shown to underlie metrologists' approaches for tackling the problem of epistemic circularity in physical measurement, illustrated in the special cases of measurement coordination and calibration. The article then explicates psychology's challenges for establishing genuine analogues of measurement, which arise from the peculiarities of its study phenomena (e.g., higher-order complexity, non-ergodicity) and language-based methods (e.g., inbuilt semantics). It demonstrates that psychometrics cannot establish coordinated and calibrated modelling relations, thus generating only pragmatic quantifications with predictive power but precluding epistemically justified inferences on the phenomena studied. This epistemic gap is often overlooked, however, because many psychologists mistake their methods' inbuilt semantics—thus, descriptions of their study phenomena (e.g., in rating scales, item variables, statistical models)—for the phenomena described. This blurs the epistemically necessary distinction between the phenomena studied and those used as means of investigation, thereby confusing ontological with epistemological concepts—psychologists' cardinal error. Therefore, many mistake judgements of verbal statements for measurements of the phenomena described and overlook that statistics can neither establish nor analyze a model's relations to the phenomena explored. The article elaborates epistemological and methodological fundamentals to establish coherent modelling relations between real and formal study system and to distinguish the epistemic components involved, considering psychology's peculiarities. It shows that epistemically justified inferences necessitate methods for analysing individuals' unrestricted verbal responses, now advanced through artificial intelligence systems modelling natural language (e.g., NLP algorithms, LLMs). Their increasing use to generate standardised descriptions of study phenomena for rating scales and constructs, by contrast, will only perpetuate psychologists' cardinal error—and thus, psychology's crisis.

Item Type:	Article
Additional Information:	This article is part of the Research Topic "Critical Debates on Quantitative Psychology and Measurement: Revived and Novel Perspectives on Fundamental Problems."
Uncontrolled Keywords:	measurement, psychometrics, large language models (LLMs), natural language processing (NLP), rating scales, modelling relation, semantics, syntax, metrology, repeatability, replicability, psychology, crisis, validity, generalizability, confidence
Subjects:	B Philosophy. Psychology. Religion > BF Psychology
Faculty / School / Research Centre / Research Group:	Faculty of Education, Health & Human Sciences Faculty of Education, Health & Human Sciences > School of Human Sciences (HUM)
Related URLs:	Author Author
Last Modified:	24 Jul 2025 06:57
URI:	https://gala.gre.ac.uk/id/eprint/50862

Actions (login required)

View Item

Downloads