Phoenix: A Novel Context-Aware Voice-Powered Math Equation Workspace and Editor

2025-08-10

AI Summary

Individuals with fine motor disabilities (FMDs) experience excessive cognitive load during speech-based mathematical interaction due to the need for precise symbolic articulation and reliance on unintuitive, command-driven interfaces. Method: This paper proposes a speech-driven mathematical editing workspace that integrates neuroscientific principles with large language models (LLMs) to build a context-aware engine supporting natural-language input and situationally grounded mathematical understanding—moving beyond traditional imperative interaction paradigms. The system achieves end-to-end integration of automatic speech recognition, semantic parsing, and symbolic expression generation. Contribution/Results: The approach enables low-cognitive-load, high-accuracy mathematical expression construction and problem solving. User studies demonstrate significant improvements in input efficiency and interaction fluency for FMDs, validating the effectiveness and feasibility of natural-language–based spoken mathematical interaction in educational assistive technologies.

Technology Category

• Natural Language Processing: Speech
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Humans and AI: AI for Accessibility

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Writing mathematical notation requires substantial effort, diverting cognitive resources from conceptual understanding to documentation mechanics, significantly impacting individuals with fine motor disabilities (FMDs). Current limits of speech-based math technologies rely on precise dictation of math symbols and unintuitive command-based interfaces. We present a novel voice-powered math workspace, applying neuroscience insights to create an intuitive problem-solving environment. To minimize cognitive load, we leverage large language models with our novel context engine to support natural language interaction. Ultimately, we enable fluid mathematical engagement for individuals with FMDs -- freed from mechanical constraints.

Problem

• Reduces cognitive load in math notation writing
• Improves voice-based math input for motor disabilities
• Enhances natural language interaction in math editing

Innovation

• Context-aware voice-powered math workspace
• Natural language interaction via LLMs
• Intuitive problem-solving environment design