PACE: Prune-And-Compress Ensemble Models

2026-05-07

AI Summary

While ensemble models achieve strong performance, their large scale poses significant challenges in deployment, interpretability, and robustness verification. This work proposes PACE, a novel framework that uniquely integrates model compression and pruning through a two-stage alternating strategy. First, it enhances ensemble diversity by generating a set of diverse weak learners guided by a theoretically grounded active learning mechanism. Subsequently, it applies fidelity-aware pruning to the expanded ensemble, enabling controlled preservation of the original model’s behavioral characteristics. Extensive experiments demonstrate that PACE consistently outperforms existing compression and pruning methods across multiple tasks, achieving higher compression ratios while more faithfully retaining the behavior of the original ensemble.

Abstract

Ensemble models achieve state-of-the-art performance on prediction tasks, but usually require aggregating a large number of weak learners. This can hinder deployment, interpretability, and downstream tasks such as robustness verification. Remedies to this issue fall into two main camps: pruning, which discards redundant learners, and compression, which generates new ones from scratch. We introduce PACE, a framework that interleaves these paradigms in a two-phase strategy. First, new learners are actively generated via a theoretically grounded procedure to enhance the diversity of the initial ensemble. When no more relevant learners can be found, a second phase of pruning is performed on this enriched ensemble. During both operations, PACE allows fine control on the faithfulness to the original ensemble. Experiments show that our method outperforms prior pruning and compression methods while offering principled control of faithfulness guarantees.

Problem

• ensemble models
• pruning
• compression
• deployment
• interpretability

Innovation

• ensemble compression
• model pruning
• diversity enhancement
• faithfulness guarantee
• active learner generation