A Lexical Analysis of online Reviews on Human-AI Interactions

2025-11-17

AI Summary

This study addresses persistent pain points and challenges in human-AI interaction by analyzing authentic user experiences. Methodologically, it conducts the first large-scale lexical factor analysis on 55,968 cross-platform online reviews from G2, Product Hunt, and Trustpilot, integrating lexical analysis, exploratory factor analysis, and qualitative content analysis. The results identify six core dimensions impairing user experience—including lack of credibility, intent misinterpretation, and ambiguous feedback—revealing previously overlooked structural deficiencies in current AI systems. Crucially, the study delivers interpretable and reproducible empirical evidence, and proposes the first empirically grounded, user-derived factor model for human-AI interaction. This model establishes a theoretical framework and actionable methodology for user-centered AI design, evaluation, and iterative improvement.

Technology Category

• Humans and AI: User Experience and Usability
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation

Abstract

This study focuses on understanding the complex dynamics between humans and AI systems by analyzing user reviews. While previous research has explored various aspects of human-AI interaction, such as user perceptions and ethical considerations, there remains a gap in understanding the specific concerns and challenges users face. By using a lexical approach to analyze 55,968 online reviews from G2.com, Producthunt.com, and Trustpilot.com, this preliminary research aims to analyze human-AI interaction. Initial results from factor analysis reveal key factors influencing these interactions. The study aims to provide deeper insights into these factors through content analysis, contributing to the development of more user-centric AI systems. The findings are expected to enhance our understanding of human-AI interaction and inform future AI technology and user experience improvements.

Problem

• Understanding user concerns and challenges in human-AI interactions
• Analyzing lexical patterns in online reviews of AI systems
• Identifying key factors influencing human-AI interaction dynamics

Innovation

• Lexical analysis of online user reviews
• Factor analysis to reveal interaction factors
• Content analysis for user-centric AI development