Exploration-Exploitation Tradeoff in Universal Lossy Compression

2025-06-25

AI Summary

This work addresses the exploration-exploitation trade-off in sequential forward/backward adaptation for universal lossy compression, particularly under short codewords and non-stationary sources—where robustness remains a critical bottleneck. We introduce, for the first time, multi-armed bandit (MAB) modeling into the sequential adaptive process of lossy compression, proposing a cost-aware robust MAB algorithm that replaces conventional natural-type selection mechanisms. The method enables joint online learning and reconstruction optimization for arbitrary codeword lengths without requiring source priors. Theoretical analysis establishes its asymptotic optimality in non-stationary environments. Experiments demonstrate substantial improvements in both rate-distortion performance and reconstruction robustness for short-block scenarios, overcoming the limited few-shot adaptability inherent in existing approaches.

Technology Category

• Machine Learning: Online Learning & Bandits
• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Universal compression can learn the source and adapt to it either in a batch mode (forward adaptation), or in a sequential mode (backward adaptation). We recast the sequential mode as a multi-armed bandit problem, a fundamental model in reinforcement-learning, and study the trade-off between exploration and exploitation in the lossy compression case. We show that a previously proposed "natural type selection" scheme can be cast as a reconstruction-directed MAB algorithm, for sequential lossy compression, and explain its limitations in terms of robustness and short-block performance. We then derive and analyze robust cost-directed MAB algorithms, which work at any block length.

Problem

• Explores exploration-exploitation tradeoff in lossy compression
• Analyzes natural type selection as MAB algorithm limitations
• Derives robust cost-directed MAB algorithms for any block length

Innovation

• Recasts sequential mode as multi-armed bandit problem
• Proposes reconstruction-directed MAB algorithm
• Derives robust cost-directed MAB algorithms