Wavelet Policy: Lifting Scheme for Policy Learning in Long-Horizon Tasks

2025-07-06

AI Summary

To address the challenge of modeling multimodal, long-horizon action-observation sequences in vision-based control tasks, this paper proposes a multi-scale policy learning framework grounded in a differentiable wavelet lifting scheme. It is the first work to deeply integrate wavelet time-frequency analysis into deep reinforcement learning, enabling end-to-end trainable multi-resolution feature decomposition that jointly supports hierarchical state representation and cross-scale action planning. Experiments on robotic manipulation, autonomous driving, and multi-robot coordination demonstrate significant improvements in policy accuracy, execution robustness, and generalization across unseen scenarios. The core contributions are: (1) the design of the first differentiable wavelet lifting policy network, and (2) the establishment of a unified optimization paradigm that jointly learns multi-scale time-frequency representations and policy decisions. This framework bridges signal processing theory with deep RL, offering a principled approach to temporal abstraction in long-horizon decision-making.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Policy learning focuses on devising strategies for agents in embodied artificial intelligence systems to perform optimal actions based on their perceived states. One of the key challenges in policy learning involves handling complex, long-horizon tasks that require managing extensive sequences of actions and observations with multiple modes. Wavelet analysis offers significant advantages in signal processing, notably in decomposing signals at multiple scales to capture both global trends and fine-grained details. In this work, we introduce a novel wavelet policy learning framework that utilizes wavelet transformations to enhance policy learning. Our approach leverages learnable multi-scale wavelet decomposition to facilitate detailed observation analysis and robust action planning over extended sequences. We detail the design and implementation of our wavelet policy, which incorporates lifting schemes for effective multi-resolution analysis and action generation. This framework is evaluated across multiple complex scenarios, including robotic manipulation, self-driving, and multi-robot collaboration, demonstrating the effectiveness of our method in improving the precision and reliability of the learned policy.

Problem

• Handling complex long-horizon tasks in policy learning
• Enhancing policy learning with multi-scale wavelet decomposition
• Improving precision and reliability in robotic and autonomous tasks

Innovation

• Wavelet transformations enhance policy learning
• Learnable multi-scale wavelet decomposition for analysis
• Lifting schemes for multi-resolution action generation