Design Opportunities for Explainable AI Paraphrasing Tools: A User Study with Non-native English Speakers

2024-05-13

AI Summary

This study addresses the information evaluation challenges faced by non-native English speakers (NNESs) in AI-assisted paraphrasing. Methodologically, it employs human-AI interaction experiments, behavioral log analysis, and explainable AI (XAI) interface design to develop and evaluate ParaScope—a system integrating multi-source explanatory modules, including back-translation verification, interpretive textual explanations, and authentic usage examples. The work first uncovers the dynamic evolution of NNESs’ information needs with respect to language proficiency and proposes a “global-to-detail” progressive interaction paradigm. Empirical results demonstrate that multi-source explanation synergies significantly outperform single-metric (e.g., back-translation) decision support. The system substantially enhances users’ writing confidence, autonomy, and efficiency; identifies critical thresholds for explanation-induced information overload; and distills seven evidence-based design principles for explainable AI writing tools tailored to NNESs.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We investigate how non-native English speakers (NNESs) interact with diverse information aids to assess and select AI-generated paraphrases. We develop ParaScope, an AI paraphrasing assistant that integrates diverse information aids, such as back-translation, explanations, and usage examples, and logs user interaction data. Our in-lab study with 22 NNESs reveals that user preferences for information aids vary by language proficiency, with workflows progressing from global to more detailed information. While back-translation was the most frequently used aid, it was not a decisive factor in suggestion acceptance; users combined multiple information aids to make informed decisions. Our findings demonstrate the potential of explainable AI paraphrasing tools to enhance NNESs' confidence, autonomy, and writing efficiency, while also emphasizing the importance of thoughtful design to prevent information overload. Based on these findings, we offer design implications for explainable AI paraphrasing tools that support NNESs in making informed decisions when using AI writing systems.

Problem

• How non-native English speakers use AI paraphrasing tools
• Designing explainable AI aids for informed paraphrase selection
• Balancing information aids to prevent user overload

Innovation

• ParaScope integrates back-translation, explanations, and examples
• User preferences vary by language proficiency levels
• Combining multiple aids enhances decision-making confidence