Safe Multi-Agent Behavior Must Be Maintained, Not Merely Asserted: Constraint Drift in LLM-Based Multi-Agent Systems

2026-05-11

AI Summary

This work addresses a critical yet overlooked issue in large language model (LLM) multi-agent systems: safety constraints are often lost, weakened, or distorted during memory retention, inter-agent communication, or tool invocation, potentially leading to privacy breaches, unauthorized actions, or non-auditable behavior—even when final outputs appear compliant. To tackle this, the paper introduces the novel concept of “constraint drift” and argues that safe behavior must be dynamically maintained rather than assumed from initial declarations. It proposes a constraint-centric governance framework that explicitly preserves constraints throughout execution via constraint state modeling, constraint-native reinforcement learning, constraint inheritance mechanisms, and auditable execution traces. This approach ensures that safety constraints remain maintainable, inheritable, enforceable, and verifiable, substantially enhancing system security and reliability in complex workflows.

Abstract

Modern LLM based agents are no longer passive text generators. They read repositories, call tools, browse the web, execute code, maintain memory, communicate with other agents, and act through long horizon workflows. This shift moves the unit of safety. A system may produce a compliant final answer while leaking private information through an internal message, delegating authority beyond its original scope, calling an external tool with sensitive context, or losing the evidence needed to reconstruct why an action was allowed. We argue that many emerging failures in LLM-based multi-agent systems share a common structure: safety critical constraints do not remain operative throughout the trajectory. We call this phenomenon constraint drift: the loss, distortion, weakening, or relaxation of constraints as they pass through memory, delegation, communication, tool use, audit, and optimization. The position taken here is that safe multi-agent behavior must be maintained, not merely asserted. Prompts, guardrails, tool schemas, access control, and final output checks are necessary, but they are insufficient unless constraints remain fresh, inherited, enforceable, and auditable across execution. We propose Constraint State Governance as a research paradigm for LLM-based multi-agent systems. In this paradigm, safety-critical constraints are maintained as explicit execution state, while constraint-native reinforcement learning improves utility only within maintained safety boundaries. The goal is not to freeze agentic systems under rigid rules, but to make safety operational across the trajectories through which modern agents actually act.

Problem

• constraint drift
• multi-agent systems
• LLM safety
• execution trajectory
• safety constraints

Innovation

• Constraint Drift
• Constraint State Governance
• Multi-Agent Safety
• LLM-Based Agents
• Safety-Critical Constraints