PrivAct: Internalizing Contextual Privacy Preservation via Multi-Agent Preference Training

2026-02-14

AI Summary

This work addresses the vulnerability of large language model (LLM) agents to inadvertently leaking sensitive information during personalized tasks due to the implicit nature of contextual privacy. Existing approaches rely on external interventions, which suffer from fragility and limited applicability across scenarios. To overcome these limitations, we propose PrivAct, a novel framework that internalizes contextual privacy preferences within multi-agent systems for the first time. PrivAct enables agents to autonomously adhere to privacy norms during behavior generation through multi-agent preference training, explicit privacy modeling, and joint optimization of privacy and utility. The framework is compatible with diverse LLM architectures and achieves up to a 12.32% reduction in privacy leakage across multiple benchmarks while preserving task helpfulness. Furthermore, it demonstrates strong zero-shot generalization and robustness to varying multi-agent topologies.

Technology Category

• Machine Learning: Privacy
• Multiagent Systems: Adversarial Agents
• Humans and AI: Learning Human Values and Preferences

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Large language model (LLM) agents are increasingly deployed in personalized tasks involving sensitive, context-dependent information, where privacy violations may arise in agents'action due to the implicitness of contextual privacy. Existing approaches rely on external, inference-time interventions which are brittle, scenario-specific, and may expand the privacy attack surface. We propose PrivAct, a contextual privacy-aware multi-agent learning framework that internalizes contextual privacy preservation directly into models'generation behavior for privacy-compliant agentic actions. By embedding privacy preferences into each agent, PrivAct enhances system-wide contextual integrity while achieving a more favorable privacy-helpfulness tradeoff. Experiments across multiple LLM backbones and benchmarks demonstrate consistent improvements in contextual privacy preservation, reducing leakage rates by up to 12.32% while maintaining comparable helpfulness, as well as zero-shot generalization and robustness across diverse multi-agent topologies. Code is available at https://github.com/chengyh23/PrivAct.

Problem

• contextual privacy
• LLM agents
• privacy violation
• multi-agent systems
• privacy preservation

Innovation

• contextual privacy
• multi-agent learning
• privacy-preserving LLMs
• preference training
• privacy-helpfulness tradeoff