Pathway-based Progressive Inference (PaPI) for Energy-Efficient Continual Learning

2025-06-21

AI Summary

Continual learning under resource-constrained settings faces dual challenges of catastrophic forgetting and high energy consumption. Method: This paper proposes Pathwise Progressive Inference (PaPI), a novel framework that jointly addresses these issues by formalizing path selection within the continual learning theoretical framework. PaPI employs Fisher-information-driven dynamic routing and energy-constrained optimization to explicitly balance model stability and plasticity. Contribution/Results: We provide theoretical guarantees showing that both forgetting rate and energy consumption converge at O(K) complexity, outperforming EWC and GEM in both forgetting mitigation and energy efficiency. Extensive experiments across multiple benchmarks demonstrate that PaPI significantly reduces computational energy consumption while effectively alleviating catastrophic forgetting. PaPI establishes a new paradigm for low-power continual learning—rigorous in theory and practical in implementation.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments

Abstract

Continual learning systems face the dual challenge of preventing catastrophic forgetting while maintaining energy efficiency, particularly in resource-constrained environments. This paper introduces Pathway-based Progressive Inference (PaPI), a novel theoretical framework that addresses these challenges through a mathematically rigorous approach to pathway selection and adaptation. We formulate continual learning as an energy-constrained optimization problem and provide formal convergence guarantees for our pathway routing mechanisms. Our theoretical analysis demonstrates that PaPI achieves an $mathcal{O}(K)$ improvement in the stability-plasticity trade-off compared to monolithic architectures, where $K$ is the number of pathways. We derive tight bounds on forgetting rates using Fisher Information Matrix analysis and prove that PaPI's energy consumption scales with the number of active parameters rather than the total model size. Comparative theoretical analysis shows that PaPI provides stronger guarantees against catastrophic forgetting than Elastic Weight Consolidation (EWC) while maintaining better energy efficiency than both EWC and Gradient Episodic Memory (GEM). Our experimental validation confirms these theoretical advantages across multiple benchmarks, demonstrating PaPI's effectiveness for continual learning in energy-constrained settings. Our codes are available at https://github.com/zser092/PAPI_FILES.

Problem

• Prevent catastrophic forgetting in continual learning systems
• Maintain energy efficiency in resource-constrained environments
• Optimize pathway selection for stability-plasticity trade-off

Innovation

• Pathway-based Progressive Inference for continual learning
• Energy-constrained optimization with formal guarantees
• Fisher Information Matrix bounds forgetting rates