Unveiling User Perceptions in the Generative AI Era: A Sentiment-Driven Evaluation of AI Educational Apps' Role in Digital Transformation of e-Teaching

2025-12-12

AI Summary

This study addresses the lack of user-perception research on AI educational applications amid digital pedagogical transformation. Leveraging Google Play user reviews, it establishes a three-tier analytical framework integrating RoBERTa-based fine-grained sentiment classification, GPT-4o–driven key-point extraction, and GPT-5–enabled thematic synthesis. It presents the first cross-category efficacy comparison across seven AI tutoring tool types (e.g., homework assistance, language learning, learning management systems), revealing significantly higher positive sentiment for homework apps versus language tools (e.g., Teacher AI: 21.8% positive) and identifying high-performing tools such as Edu AI (95.9% positive). Quantitative analysis confirms AI’s benefits for problem-solving efficiency and cognitive stimulation, while exposing critical risks—including paywalls, hallucinations, and intrusive advertising. Based on these findings, the study proposes a three-stage evolution pathway: hybrid human-AI instruction, VR/AR-immersive learning, and inclusive regulatory governance.

Technology Category

• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Humans and AI: User Experience and Usability
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems

Abstract

The rapid integration of generative artificial intelligence into education has driven digital transformation in e-teaching, yet user perceptions of AI educational apps remain underexplored. This study performs a sentiment-driven evaluation of user reviews from top AI ed-apps on the Google Play Store to assess efficacy, challenges, and pedagogical implications. Our pipeline involved scraping app data and reviews, RoBERTa for binary sentiment classification, GPT-4o for key point extraction, and GPT-5 for synthesizing top positive/negative themes. Apps were categorized into seven types (e.g., homework helpers, math solvers, language tools), with overlaps reflecting multifunctional designs. Results indicate predominantly positive sentiments, with homework apps like Edu AI (95.9% positive) and Answer.AI (92.7%) leading in accuracy, speed, and personalization, while language/LMS apps (e.g., Teacher AI at 21.8% positive) lag due to instability and limited features. Positives emphasize efficiency in brainstorming, problem-solving, and engagement; negatives center on paywalls, inaccuracies, ads, and glitches. Trends show that homework helpers outperform specialized tools, highlighting AI's democratizing potential amid risks of dependency and inequity. The discussion proposes future ecosystems with hybrid AI-human models, VR/AR for immersive learning, and a roadmap for developers (adaptive personalization) and policymakers (monetization regulation for inclusivity). This underscores generative AI's role in advancing e-teaching by enabling ethical refinements that foster equitable, innovative environments. The full dataset is available here(https://github.com/erfan-nourbakhsh/GenAI-EdSent).

Problem

• Evaluates user perceptions of AI educational apps in digital teaching transformation
• Analyzes sentiment and themes from app reviews to assess efficacy and challenges
• Identifies performance gaps between different AI app types and pedagogical implications

Innovation

• Used RoBERTa for sentiment classification
• Applied GPT-4o for key point extraction
• Employed GPT-5 to synthesize review themes