Position: Capability Control Should be a Separate Goal From Alignment

2026-02-05

AI Summary

This work addresses the growing risks of misuse and loss of control associated with the broad applicability of foundation models, which existing alignment methods struggle to mitigate through hard behavioral constraints. It establishes capability control as a core objective distinct from alignment and introduces a defense-in-depth framework spanning data, learning, and system layers to enforce multi-granular behavioral constraints throughout the model lifecycle. By integrating techniques such as data distribution shaping, representational intervention, and runtime input/output/action-level safeguards, the paper systematically constructs pathways for capability control. It further identifies critical challenges—including the dual-use nature of knowledge and combinatorial generalization—offering a new paradigm for developing safe and controllable AI systems.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Large Vision Models
• Natural Language Processing: Safety and Robustness

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Foundation models are trained on broad data distributions, yielding generalist capabilities that enable many downstream applications but also expand the space of potential misuse and failures. This position paper argues that capability control -- imposing restrictions on permissible model behavior -- should be treated as a distinct goal from alignment. While alignment is often context and preference-driven, capability control aims to impose hard operational limits on permissible behaviors, including under adversarial elicitation. We organize capability control mechanisms across the model lifecycle into three layers: (i) data-based control of the training distribution, (ii) learning-based control via weight- or representation-level interventions, and (iii) system-based control via post-deployment guardrails over inputs, outputs, and actions. Because each layer has characteristic failure modes when used in isolation, we advocate for a defense-in-depth approach that composes complementary controls across the full stack. We further outline key open challenges in achieving such control, including the dual-use nature of knowledge and compositional generalization.

Problem

• capability control
• foundation models
• misuse prevention
• adversarial elicitation
• operational limits

Innovation

• capability control
• defense-in-depth
• foundation models
• alignment
• compositional generalization