Spectral Rewiring for Exploration, Purification, and Model Merging

2026-07-03

AI Summary

This work addresses the challenges of reasoning performance saturation and cross-domain interference commonly encountered during reinforcement learning-based post-training of large language models. The authors propose Subspace-Aligned Rewiring (SAR), a novel method that reveals— for the first time—that effective reasoning updates are confined to a specific subspace within the base model’s spectral space. Leveraging this insight, SAR performs post-hoc model editing to retain critical components while removing orthogonal, interfering directions. Remarkably, the approach extracts a compact reasoning core using only approximately 0.58% of the original parameters, preserving over 99% of post-training performance while significantly enhancing mathematical reasoning capabilities. Evaluations on seven internal coding benchmarks show improvements on six, and the method achieves superior cross-domain fusion performance compared to single-domain expert models.

Abstract

Reinforcement learning has become a standard post-training recipe for large language models, but dense full-parameter updates create two deployment-relevant bottlenecks: suppressed reasoning performance, often reflected by premature saturation of test-time scaling, and interference when consolidating multiple capabilities through multi-domain training or model merging. We show that the reasoning-effective component of these updates is largely concentrated in the base model's spectral space, motivating Subspace-Aligned Rewiring (SAR), a post-hoc editing method that retains this spectral core while removing orthogonal components. SAR therefore preserves reasoning gains and filters residual update directions that suppress performance or amplify cross-domain interference. Across several model families and scales, SAR extracts compact reasoning cores using as little as approximately 0.58% of total parameters: it preserves over 99% of post-training performance and improves high-k exploration in mathematical reasoning, and generalizes to agentic coding by improving six of seven open benchmarks on an in-house model. SAR also purifies mixed-domain training updates by releasing suppressed coding capability while maintaining math reasoning and instruction following. It further enables model merging across experts, yielding cross-domain generalization that surpasses previous merging baselines and even the best single-domain experts. Overall, SAR shows that extracting reasoning-effective updates from parameter geometry can serve as a training-free mechanism to improve reasoning and multi-domain performance.

Problem

• reinforcement learning
• reasoning performance
• parameter interference
• model merging
• multi-domain training

Innovation

• spectral rewiring
• Subspace-Aligned Rewiring
• reasoning core
• model merging
• parameter geometry