LoMC: Localized Multidirectional Correction for Refusal Suppression in Routed Foundation Models

2026-06-10

AI Summary

Existing refusal suppression methods for routed and hybrid Mixture-of-Experts (MoE) models either apply overly broad interventions that degrade general capabilities or restrict edits to supporting experts alone, resulting in insufficient correction. To address this, we propose the Localized Multidirectional Correction (LoMC) framework, which identifies a compact edit-support set and generates layer-wise low-rank corrections by aggregating multidirectional prototypes. LoMC further incorporates a support-gating mechanism to apply interventions precisely within the support set, thereby avoiding unnecessary modifications elsewhere. This approach significantly enhances correction capacity without expanding the intervention scope. Experiments demonstrate that LoMC effectively strengthens non-refusal target response behaviors across four routed backbone models and multimodal safety benchmarks, while preserving model generalization with minimal intervention footprint.

Abstract

We study controlled post-training refusal suppression in routed MoE and hybrid-MoE foundation models, aiming to increase non-refusal target-response behavior while preserving general capability under a compact intervention footprint. Existing broad direction-based edits can perturb general-purpose computation, whereas support-only expert edits often lack sufficient capacity to correct heterogeneous refusal representations. To address this limitation, we introduce Localized Multidirectional Correction (LoMC), a support-gated intervention framework that follows a support-then-correction execution order: it first identifies a compact edit support, then aggregates prototype correction directions into layer-wise correction directions, and finally applies rank-one layer-wise correction only within the selected support. By using the edit support as a structural gating constraint, LoMC increases correction capacity without expanding the intervention scope. Experiments on text-only and multimodal safety benchmarks across four routed backbones show that LoMC substantially improves non-refusal target-response behavior while maintaining general capability under a compact intervention footprint.

Problem

• refusal suppression
• routed foundation models
• mixture of experts
• controlled post-training
• intervention footprint

Innovation

• Localized Multidirectional Correction
• refusal suppression
• routed MoE
• support-gated intervention
• compact intervention footprint