Component-Aware Pruning Framework for Neural Network Controllers via Gradient-Based Importance Estimation

2026-01-27

AI Summary

This work addresses the challenge of deploying multi-component neural network controllers, which are often hindered by high computational complexity, and the inadequacy of conventional norm-based pruning methods in accurately capturing the functional importance of individual components. To this end, the paper introduces a component-aware structured pruning framework that, for the first time, integrates three gradient-driven importance metrics—gradient accumulation, Fisher information, and Bayesian uncertainty—into the pruning of multi-component controllers. These metrics enable dynamic assessment of component importance during training, uncovering structural dependencies and temporal variations overlooked by static heuristic approaches. Experiments on autoencoders and TD-MPC reinforcement learning agents demonstrate that the proposed method more accurately identifies critical components, achieving substantial model compression while effectively preserving performance.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Learning to Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

The transition from monolithic to multi-component neural architectures in advanced neural network controllers poses substantial challenges due to the high computational complexity of the latter. Conventional model compression techniques for complexity reduction, such as structured pruning based on norm-based metrics to estimate the relative importance of distinct parameter groups, often fail to capture functional significance. This paper introduces a component-aware pruning framework that utilizes gradient information to compute three distinct importance metrics during training: Gradient Accumulation, Fisher Information, and Bayesian Uncertainty. Experimental results with an autoencoder and a TD-MPC agent demonstrate that the proposed framework reveals critical structural dependencies and dynamic shifts in importance that static heuristics often miss, supporting more informed compression decisions.

Problem

• neural network controllers
• model compression
• structured pruning
• computational complexity
• parameter importance

Innovation

• component-aware pruning
• gradient-based importance
• structured model compression
• neural network controllers
• dynamic importance estimation