Selected work

Representative Papers

A Theory of Conditional Collapse under Low-Rank Weight-Space Ablations: I. The Single-Block Theory and Synthetic Validation

Aug 04, 2026

This study investigates the conditions under which activation patching and weight-space ablation yield consistent assessments of the causal influence of model components. By constructing an idealized additive residual stream model and integrating low-rank weight ablation, first-order interaction expansions, and synthetic task validation, the work establishes the first theoretical criterion for their consistency. The analysis reveals fundamental differences in how the two methods affect readout mechanisms, derives precise error expressions and interaction formulas, and empirically validates three core predictions on synthetic tasks: ablation configurations exhibit a strong negative correlation with model accuracy (Spearman ρ = −0.83), and polarity reversal phenomena are reproducible across distinct architectures.

0 citationsRead paper

Cross-Layer Interaction under Weight-Space Ablation: A Closed-Form Attention Jacobian Bound and a Test on a Real Pretrained Model

Aug 04, 2026

This work investigates the discrepancy between weight ablation and activation patching outcomes in real pretrained models, extending residual block interaction analysis to multi-layer settings. By decomposing interactions via double integrals into intra-block terms and cross-layer remainders, it derives—for the first time—a closed-form upper bound on the Jacobian norm of attention submodules, explicitly characterizing the curvature constant. Combining weight ablation, activation patching, and mixed second-derivative analysis, the study validates that the theoretical bounds hold without violation across all tested cases on Qwen2.5-1.5B-Instruct. Furthermore, it identifies a three-layer indirect object identification circuit shared across five diverse instances, revealing substantial cross-layer interactions that transcend the limitations of single-block theoretical frameworks.

0 citationsRead paper

Exact Network Surgery: Functional Invariance and Gradient Plasticity in Reactive Computational Graphs

Jul 17, 2026

This work addresses the challenge that existing network expansion methods struggle to simultaneously preserve bit-exact functional outputs and enable immediate trainability of newly introduced parameters. We propose Exact Network Surgery, which achieves both goals for the first time by inserting gated residual blocks in-place within the runtime computation graph, ensuring strictly unchanged outputs while instantly activating gradients for new parameters. Theoretically, we establish the Identity Morphism Theorem, Structural Locality Theorem, and Zero-Initialization Escape Proposition, and identify a class of non-escapable degenerate saddle points. Implemented atop the NeuroDSL reactive graph engine (in Julia), experiments confirm 1,600 logit outputs with zero numerical error, immediate departure of gated parameters from zero initialization, persistently zero gradients under degenerate configurations, and surgery overhead highly correlated with downstream cone size (r = 0.9992), enabling bit-exact training resumption after interruption.

0 citationsRead paper

Cost Accounting for Reactive Computational Graphs: Exhaustive Sweeps, Sequential Mutation, and the Backward-Locality Gap

Jul 17, 2026

This work addresses the prohibitive computational overhead of systematic interventions—such as activation patching and circuit discovery—in neural network computation graphs, which stems from frequent recomputation. Building upon the reactive graph engine NeuroDSL (implemented in Julia), the authors introduce the first exact cost model for exhaustive interventions by integrating graph invalidation theory, Karamata index analysis, and closed-form combinatorial counting. The model yields a depth-dependent closed-form solution for scan speedup, a formula quantifying redundant computation in persistent mutation sequences, and a rigorous proof of locality failure in backpropagation. Empirical validation across four cost regimes confirms that scan speedup converges to its theoretical limit, training-mode speedup collapses to unity, and measured costs for 18 distinct mutation operations match predictions with zero error.

0 citationsRead paper

EPIG: Emotion-Based Prompting for Personalised Image Generation

Jun 11, 2026

Current text-to-image generation models exhibit limitations in conveying nuanced emotional intent. This work proposes a lightweight, training-free emotion-aware prompting framework that, without modifying or fine-tuning the underlying diffusion model, integrates psychological valence-arousal emotion representation with character-aware mechanisms for the first time. By leveraging structured prompt templates and semantic expansion, the approach enhances emotional expressiveness while preserving semantic fidelity. Evaluated across ten diverse prompt benchmarks, the method significantly outperforms strong baselines, reducing average arousal prediction error by 12%–17% while maintaining alignment in valence and overall semantic consistency, thereby enabling more emotionally coherent personalized image generation.

0 citationsRead paper

Recent publications

Latest Papers

A Theory of Conditional Collapse under Low-Rank Weight-Space Ablations: I. The Single-Block Theory and Synthetic Validation

Aug 04, 2026

This study investigates the conditions under which activation patching and weight-space ablation yield consistent assessments of the causal influence of model components. By constructing an idealized additive residual stream model and integrating low-rank weight ablation, first-order interaction expansions, and synthetic task validation, the work establishes the first theoretical criterion for their consistency. The analysis reveals fundamental differences in how the two methods affect readout mechanisms, derives precise error expressions and interaction formulas, and empirically validates three core predictions on synthetic tasks: ablation configurations exhibit a strong negative correlation with model accuracy (Spearman ρ = −0.83), and polarity reversal phenomena are reproducible across distinct architectures.

0 citationsRead paper

Cross-Layer Interaction under Weight-Space Ablation: A Closed-Form Attention Jacobian Bound and a Test on a Real Pretrained Model

Aug 04, 2026

This work investigates the discrepancy between weight ablation and activation patching outcomes in real pretrained models, extending residual block interaction analysis to multi-layer settings. By decomposing interactions via double integrals into intra-block terms and cross-layer remainders, it derives—for the first time—a closed-form upper bound on the Jacobian norm of attention submodules, explicitly characterizing the curvature constant. Combining weight ablation, activation patching, and mixed second-derivative analysis, the study validates that the theoretical bounds hold without violation across all tested cases on Qwen2.5-1.5B-Instruct. Furthermore, it identifies a three-layer indirect object identification circuit shared across five diverse instances, revealing substantial cross-layer interactions that transcend the limitations of single-block theoretical frameworks.

0 citationsRead paper

Exact Network Surgery: Functional Invariance and Gradient Plasticity in Reactive Computational Graphs

Jul 17, 2026

This work addresses the challenge that existing network expansion methods struggle to simultaneously preserve bit-exact functional outputs and enable immediate trainability of newly introduced parameters. We propose Exact Network Surgery, which achieves both goals for the first time by inserting gated residual blocks in-place within the runtime computation graph, ensuring strictly unchanged outputs while instantly activating gradients for new parameters. Theoretically, we establish the Identity Morphism Theorem, Structural Locality Theorem, and Zero-Initialization Escape Proposition, and identify a class of non-escapable degenerate saddle points. Implemented atop the NeuroDSL reactive graph engine (in Julia), experiments confirm 1,600 logit outputs with zero numerical error, immediate departure of gated parameters from zero initialization, persistently zero gradients under degenerate configurations, and surgery overhead highly correlated with downstream cone size (r = 0.9992), enabling bit-exact training resumption after interruption.

0 citationsRead paper

Cost Accounting for Reactive Computational Graphs: Exhaustive Sweeps, Sequential Mutation, and the Backward-Locality Gap

Jul 17, 2026

This work addresses the prohibitive computational overhead of systematic interventions—such as activation patching and circuit discovery—in neural network computation graphs, which stems from frequent recomputation. Building upon the reactive graph engine NeuroDSL (implemented in Julia), the authors introduce the first exact cost model for exhaustive interventions by integrating graph invalidation theory, Karamata index analysis, and closed-form combinatorial counting. The model yields a depth-dependent closed-form solution for scan speedup, a formula quantifying redundant computation in persistent mutation sequences, and a rigorous proof of locality failure in backpropagation. Empirical validation across four cost regimes confirms that scan speedup converges to its theoretical limit, training-mode speedup collapses to unity, and measured costs for 18 distinct mutation operations match predictions with zero error.

0 citationsRead paper

EPIG: Emotion-Based Prompting for Personalised Image Generation

Jun 11, 2026

Current text-to-image generation models exhibit limitations in conveying nuanced emotional intent. This work proposes a lightweight, training-free emotion-aware prompting framework that, without modifying or fine-tuning the underlying diffusion model, integrates psychological valence-arousal emotion representation with character-aware mechanisms for the first time. By leveraging structured prompt templates and semantic expansion, the approach enhances emotional expressiveness while preserving semantic fidelity. Evaluated across ten diverse prompt benchmarks, the method significantly outperforms strong baselines, reducing average arousal prediction error by 12%–17% while maintaining alignment in valence and overall semantic consistency, thereby enabling more emotionally coherent personalized image generation.

0 citationsRead paper