Reinforcement learning with learned gadgets to tackle hard quantum problems on real hardware

2024-10-31

AI Summary

Approximating the ground state of NP-hard Hamiltonians on near-term quantum hardware remains challenging due to limited qubit coherence, connectivity, and gate fidelity. Method: This paper introduces Gadget Reinforcement Learning (GRL), the first framework that embeds learnable, reusable composite quantum gates (“gadgets”) into the reinforcement learning action space—unifying quantum program synthesis with hardware-aware optimization. GRL integrates Proximal Policy Optimization (PPO), parameterized quantum circuits, and hardware-aware compilation. Contributions/Results: Evaluated on the transverse-field Ising model, GRL achieves significant improvements in ground-state approximation accuracy and generalization under tight resource budgets—requiring only 2–3 days of GPU training. It scales to larger problem instances and autonomously synthesizes hardware-customized, reusable circuit modules. The framework bridges algorithmic expressivity and physical constraints, enabling efficient, adaptive quantum circuit design. Open-source implementation is provided.

Technology Category

• Machine Learning: Quantum Machine Learning
• Search and Optimization: Learning to Search
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Designing quantum circuits for specific tasks is challenging due to the exponential growth of the state space. We introduce gadget reinforcement learning (GRL), which integrates reinforcement learning with program synthesis to automatically generate and incorporate composite gates (gadgets) into the action space. This enhances the exploration of parameterized quantum circuits (PQCs) for complex tasks like approximating ground states of quantum Hamiltonians, an NP-hard problem. We evaluate GRL using the transverse field Ising model under typical computational budgets (e.g., 2- 3 days of GPU runtime). Our results show improved accuracy, hardware compatibility and scalability. GRL exhibits robust performance as the size and complexity of the problem increases, even with constrained computational resources. By integrating gadget extraction, GRL facilitates the discovery of reusable circuit components tailored for specific hardware, bridging the gap between algorithmic design and practical implementation. This makes GRL a versatile framework for optimizing quantum circuits with applications in hardware-specific optimizations and variational quantum algorithms. The code is available at: https://github.com/Aqasch/Gadget_RL

Problem

• Automates quantum circuit design using reinforcement learning
• Solves NP-hard quantum ground state approximation
• Enhances hardware compatibility and scalability

Innovation

• Reinforcement learning integrates program synthesis
• Automatically generates composite quantum gates
• Enhances exploration of parameterized quantum circuits