Self-Improving is Often Sudden: Enlightenment-style Finetuning for Large-Scale Models

2026-07-14

AI Summary

Inspired by the human phenomenon of “insight,” this work proposes Enlightenment, a training-free post-tuning paradigm designed to elicit abrupt, emergent capability improvements in large-scale models. Moving beyond existing training-free approaches that solely adjust attention weights, Enlightenment introduces an architecture-aware shortcut rewiring mechanism—specifically, attention head mixing shortcuts for large language models (LLMs) and scalar modulation of residual connections for vision-language models (VLMs). Experimental results demonstrate that Enlightenment consistently achieves significant performance gains across diverse models and benchmarks, effectively unlocking latent capabilities embedded in pretrained architectures without additional training.

Abstract

The pursuit of autonomously self-improving models has attracted growing interest in the era of large-scale foundation models. Drawing inspiration from the concept of "enlightenment" or "aha moment" in human brain, we hypothesize that large models exhibit an analogous enlightenment phenomenon-a latent capacity for sudden capability boost. Then, we propose Enlightenment, a novel training-free post-tuning paradigm for large-scale models. Our approach modifies shortcuts for key modules/layers without weight updates, while existing training-free ones predominantly manipulate attention weights. We introduce two architecture-specific instantiations: i) For large language models, we propose attention head-mixing shortcuts that recalibrate attention weights by linking the initial attention head's output to all other target heads, modulated by an adaptive scaling factor initialization strategy. ii) For vision-language models, we apply a lightweight scalar-modulated factor to residual connections in the decoder layers, regulating information flow. Extensive experiments show that Enlightenment efficiently unlocks the latent potential of pre-trained networks, yielding remarkable performance improvements across diverse benchmarks and models.

Problem

• self-improving
• enlightenment
• large-scale models
• training-free
• latent capability

Innovation

• enlightenment
• training-free tuning
• shortcut modification
• attention head-mixing
• residual modulation