QuCoWE Quantum Contrastive Word Embeddings with Variational Circuits for NearTerm Quantum Devices

2025-11-13

AI Summary

To address the challenges of training and generalizing quantum word embeddings on noisy intermediate-scale quantum (NISQ) devices, this paper proposes the Quantum Contrastive Embedding (QCE) framework. QCE employs shallow variational quantum circuits incorporating data re-uploading and ring-shaped controlled-entanglement encoding for vocabulary representation. Word similarity is modeled via quantum state fidelity, and a logit-fidelity output head is designed to align the learning objective with the shifted PMI scale implicitly optimized by Skip-Gram with Negative Sampling (SGNS). To mitigate gradient vanishing and enhance trainability under noise, we introduce an entanglement-budget regularization based on single-qubit purity. Additionally, error mitigation techniques—including zero-noise extrapolation and randomized compiling—are integrated. On Text8 and WikiText2, QCE achieves performance comparable to 50–100-dimensional classical embeddings on intrinsic (WordSim353, SimLex999) and extrinsic (SST-2, TREC-6) tasks, using significantly fewer parameters per token—demonstrating hardware efficiency and semantic consistency.

Technology Category

• Machine Learning: Quantum Machine Learning
• Natural Language Processing: Learning & Optimization for NLP
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

We present QuCoWE a framework that learns quantumnative word embeddings by training shallow hardwareefficient parameterized quantum circuits PQCs with a contrastive skipgram objective Words are encoded by datareuploading circuits with controlled ring entanglement similarity is computed via quantum state fidelity and passed through a logitfidelity head that aligns scores with the shiftedPMI scale of SGNSNoiseContrastive Estimation To maintain trainability we introduce an entanglementbudget regularizer based on singlequbit purity that mitigates barren plateaus On Text8 and WikiText2 QuCoWE attains competitive intrinsic WordSim353 SimLex999 and extrinsic SST2 TREC6 performance versus 50100d classical baselines while using fewer learned parameters per token All experiments are run in classical simulation we analyze depolarizingreadout noise and include errormitigation hooks zeronoise extrapolation randomized compiling to facilitate hardware deployment

Problem

• Develop quantum-native word embeddings using variational circuits for near-term quantum devices
• Mitigate barren plateaus through entanglement-budget regularization during quantum training
• Enable hardware deployment with noise analysis and error-mitigation techniques

Innovation

• Trains shallow hardware-efficient parameterized quantum circuits
• Encodes words via data-reuploading with controlled entanglement
• Uses quantum state fidelity with logit-fidelity head alignment