Better Assumptions, Stronger Conclusions: The Case for Ordinal Regression in HCI

2026-02-20

AI Summary

This study addresses the widespread yet often inappropriate use of parametric or nonparametric statistical methods in Human-Computer Interaction (HCI) research when analyzing ordinal-scale data, such as Likert-type responses, without adequately considering the underlying assumptions about data structure. The paper systematically critiques the limitations of current analytical practices and, for the first time in the HCI field, advocates for and promotes the adoption of cumulative link models (CLMs) and their mixed-effects extensions (CLMMs) as more principled approaches to modeling ordinal outcomes. By exposing critical flaws in conventional methods’ assumptions and providing reproducible R-based examples alongside open-source datasets, this work establishes a rigorous statistical framework tailored to ordinal data, thereby substantially enhancing analytical validity and advancing methodological standardization in HCI research.

Technology Category

• Cognitive Modeling & Cognitive Systems: Analogy
• Machine Learning: Learning Preferences or Rankings
• Application Domains: Humanities & Computational Social Science

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Despite the widespread use of ordinal measures in HCI, such as Likert-items, there is little consensus among HCI researchers on the statistical methods used for analysing such data. Both parametric and non-parametric methods have been extensively used within the discipline, with limited reflection on their assumptions and appropriateness for such analyses. In this paper, we examine recent HCI works that report statistical analyses of ordinal measures. We highlight prevalent methods used, discuss their limitations and spotlight key assumptions and oversights that diminish the insights drawn from these methods. Finally, we champion and detail the use of cumulative link (mixed) models (CLM/CLMM) for analysing ordinal data. Further, we provide practical worked examples of applying CLM/CLMMs using R to published open-sourced datasets. This work contributes towards a better understanding of the statistical methods used to analyse ordinal data in HCI and helps to consolidate practices for future work.

Problem

• ordinal data
• statistical methods
• HCI
• Likert scales
• methodological assumptions

Innovation

• ordinal regression
• cumulative link model
• Likert-scale data
• statistical assumptions
• HCI methodology