Score
Modeling and detecting user or agent intent from inputs to classify, parameterize, and filter actions, including detecting adversarial intents and aligning retrieved content to fine-grained intent attributes.
Traditional unimodal text-based intent recognition suffers from limited contextual expressiveness, while human–computer interaction increasingly demands robust integration of heterogeneous signals. This paper systematically surveys deep learning–based multimodal intent recognition, focusing on synergistic modeling of textual, audio, visual, and physiological modalities. It traces the technical evolution from unimodal baselines to cross-modal fusion, emphasizing breakthrough applications of Transformer architectures in cross-modal alignment, feature fusion, and representation learning. We catalog 12 mainstream multimodal datasets, unify evaluation metrics, and identify representative application scenarios. A three-dimensional taxonomy—spanning modality combinations, fusion levels (early/late/hybrid), and learning paradigms (supervised/self-supervised/few-shot)—is proposed. Key challenges—including modality asynchrony, few-shot generalization, and model interpretability—are critically analyzed. Future directions include optimized cross-modal alignment, neuro-symbolic integration, and edge-efficient lightweight modeling, offering a structured reference for advancing multimodal intent understanding.
Resource-constrained edge devices face challenges in accurately understanding user intent from UI interaction traces, while simultaneously ensuring privacy preservation and real-time responsiveness. Method: This paper proposes a two-stage decomposed architecture: (1) generating structured sequential summaries of interaction behaviors, followed by (2) lightweight intent inference based on these summaries. The approach integrates context-aggregated enhancement and task-adaptive fine-tuning to strengthen semantic modeling capabilities of small models. Contribution/Results: Experimental results demonstrate that, under identical privacy guarantees and low-latency constraints, the proposed method achieves higher intent recognition accuracy than state-of-the-art large multimodal language models. It establishes an efficient, privacy-aware, and real-time interaction understanding paradigm for on-device intelligent agents.
Current GUI agents lack effective mechanisms for evaluating action quality, often leading to task failure due to irreversible errors. This work proposes IntentScore—a novel action-scoring model that integrates planning intent into the action encoder to distinguish between semantically similar but goal-divergent operations. Trained on 398K cross-operating-system offline interaction trajectories using contrastive learning and margin ranking loss, IntentScore learns generalizable reward signals from heterogeneous behavioral data. In held-out evaluations, it achieves a pairwise discrimination accuracy of 97.5%. When deployed as a re-ranker in the unseen environment OSWorld, it improves task success rate by 6.9 percentage points.
Intent detection suffers from strong dependence on in-domain labeled data and poor generalization to out-of-distribution (OOD) novel intents. While Generalized Intent Discovery (GID) aims to automatically discover unseen intents from unlabeled OOD data, existing approaches neglect cross-domain knowledge transfer, hindering effective reuse of prior domain knowledge for new intent identification. To address this, we propose a dual-driven paradigm integrating *prototype guidance* and *hierarchical consistency*: external prototypes preserve semantic priors from supervised models, while hierarchical consistency regularization, contrastive learning, and cross-domain feature alignment jointly enable robust knowledge transfer and novel intent discovery. Our method achieves significant improvements over state-of-the-art baselines across multiple benchmarks. The code is publicly available.
Indirect Prompt Injection Attacks (IPIAs) stealthily manipulate large language models (LLMs) into executing malicious instructions when processing untrusted inputs. Method: This paper proposes IntentGuard, a general-purpose defense framework grounded in Instruction-Intention Alignment (IIA) analysis. Instead of detecting malicious text, IntentGuard employs three cognitive intervention strategies—start-point prefilling, end-point optimization, and adversarial contextual exemplars—to explicitly probe whether reasoning-oriented LLMs (e.g., Qwen-3-32B, gpt-oss-20B) internally generate execution intentions. Contribution/Results: Evaluated on AgentDojo and Mind2Web benchmarks, IntentGuard incurs no performance degradation except in one isolated scenario, while reducing adaptive IPIA success rates from 100% to 8.5%. This demonstrates substantial improvements in agent robustness against IPIAs without compromising task fidelity.
This paper addresses a novel “intent-hiding adversarial prompting” attack against large language models (LLMs), wherein malicious intent is concealed by composing general-purpose skills to evade existing alignment mechanisms. We propose the first game-theoretic framework for modeling and analyzing this attack-defense interaction. Methodologically, we formulate a multi-stage defense game with prompt- and response-level filtering, formally characterizing the attacker’s structural advantages; we theoretically identify skill composition as the root cause of the attack’s scalability and stealth, and design corresponding defensive mechanisms grounded in this analysis. Experiments across multiple mainstream LLMs demonstrate that our attack significantly outperforms prior adversarial prompting methods, while our defense effectively mitigates intent-hiding threats and improves model robustness. The core contribution lies in establishing an interpretable, game-theoretic model of intent-hiding attacks and unifying attack analysis with principled defense design.
This work addresses the vulnerability of vision-language model–driven web agents to deceptive user interfaces, which often compromises their task execution due to insufficient defensive capabilities. To bridge this gap, the authors propose DUDE, a two-stage framework that formalizes deception-aware defense mechanisms for the first time. DUDE integrates hybrid-reward reinforcement learning, an asymmetric penalty strategy, and experience-summary distillation to distill failure patterns into transferable defensive guidance. Evaluated on RUC—a newly curated benchmark comprising 1,407 deceptive scenarios—DUDE reduces agent sensitivity to deceptive interfaces by 53.8% while preserving task performance, thereby achieving a synergistic optimization of robustness and efficacy.
This work addresses the challenges in human-AI collaborative data analysis, where rapidly evolving analytical processes often undermine shared understanding, leading to undocumented assumptions, misaligned intentions, and context-poor prompts. To mitigate these issues, the authors propose a rule-based coordination layer that explicitly models user intent as editable, structured rules and validates their consistency with shared intent in real time during prompt formulation. By integrating rule-based reasoning, notebook parsing, and structured intent representation, the approach externalizes user intentions and provides early warnings of potential conflicts. A user study demonstrates that the system significantly enhances analysts’ awareness of their collaborators’ intentions and encourages reflection on analytical strategies, offering a novel design paradigm for human-AI collaborative data analysis.
This work addresses the performance degradation of existing out-of-scope intent detection methods as the number of known intent classes increases, as well as the high computational cost and deployment challenges associated with large language model–based embeddings. To overcome these limitations, the authors propose a lightweight and efficient approach that leverages MiniLM (all-MiniLM-L6-v2) embeddings within a one-class classification framework augmented with a multi-cluster boundary learning mechanism. This mechanism explicitly models the semantic cluster structure of in-scope training utterances to better characterize the known intent distribution, thereby improving the rejection of out-of-domain samples. The method achieves state-of-the-art performance on CLINC150, StackOverflow, and Banking77 benchmarks. Ablation studies confirm the synergistic benefits of combining MiniLM embeddings with multi-cluster boundaries, demonstrating significantly enhanced out-of-scope detection while maintaining low computational overhead.
This work addresses a critical limitation in current AI interaction paradigms, which treat prompts as the primary unit of exchange while overlooking users’ underlying source intentions. The paper introduces Intent Signal Theory (IST), the first formal framework to articulate the multi-layered structure of user intent, distinguishing between source intent, intent proxies, encoded carriers, and model outputs, and establishes the Irreversible Intent Loss Theorem. By reframing prompt engineering as intent protocol design, IST reveals a missing computational layer in contemporary systems. Empirical validation across four studies, six large language models, three languages, and three task domains confirms core theoretical predictions, including structure–fidelity decoupling, metric disentanglement, and weight tolerance.
This work addresses the challenge of detecting malicious agents in multi-agent systems that evade content-based moderation by embedding harmful intent within seemingly benign interactions. To overcome the limitations of static content analysis, the authors propose Bot-Mod, a novel framework that introduces intent recognition into agent moderation through dynamic, multi-turn dialogue probing. Bot-Mod iteratively narrows down potential malicious intents in a hypothesis space using a Gibbs sampling–based dialogue strategy, enabling real-time inference of adversarial motives. The framework is evaluated on real-world community data, with experiments on the Moltbook dataset demonstrating its effectiveness in identifying diverse adversarial behaviors while maintaining a low false-positive rate for benign agents.