QiNN-QJ: A Quantum-inspired Neural Network with Quantum Jump for Multimodal Sentiment Analysis

2025-10-30

AI Summary

Existing quantum-inspired multimodal fusion models rely on a single unitary transformation to model cross-modal entanglement, resulting in training instability, poor generalization, and limited interpretability. To address these limitations, we propose the first differentiable neural network incorporating quantum jump dynamics—namely, a dissipative dynamical framework that jointly learns Hamiltonian and Lindblad operators to govern entanglement evolution across modalities. Our approach innovatively integrates quantum pure-state encoding, differentiable quantum jump operators, and steady-state attractors to guide optimization. Furthermore, we quantify entanglement via von Neumann entropy and enable posterior interpretability through trainable measurement projections. Extensive experiments on CMU-MOSI, CMU-MOSEI, and CH-SIMS demonstrate significant improvements over state-of-the-art methods, achieving superior training stability, generalization performance, and physically grounded interpretability.

Technology Category

• Machine Learning: Quantum Machine Learning
• Computer Vision: Multi-modal Vision
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Quantum theory provides non-classical principles, such as superposition and entanglement, that inspires promising paradigms in machine learning. However, most existing quantum-inspired fusion models rely solely on unitary or unitary-like transformations to generate quantum entanglement. While theoretically expressive, such approaches often suffer from training instability and limited generalizability. In this work, we propose a Quantum-inspired Neural Network with Quantum Jump (QiNN-QJ) for multimodal entanglement modelling. Each modality is firstly encoded as a quantum pure state, after which a differentiable module simulating the QJ operator transforms the separable product state into the entangled representation. By jointly learning Hamiltonian and Lindblad operators, QiNN-QJ generates controllable cross-modal entanglement among modalities with dissipative dynamics, where structured stochasticity and steady-state attractor properties serve to stabilize training and constrain entanglement shaping. The resulting entangled states are projected onto trainable measurement vectors to produce predictions. In addition to achieving superior performance over the state-of-the-art models on benchmark datasets, including CMU-MOSI, CMU-MOSEI, and CH-SIMS, QiNN-QJ facilitates enhanced post-hoc interpretability through von-Neumann entanglement entropy. This work establishes a principled framework for entangled multimodal fusion and paves the way for quantum-inspired approaches in modelling complex cross-modal correlations.

Problem

• Modeling multimodal entanglement with quantum jump dynamics
• Stabilizing training through structured stochasticity and attractors
• Enhancing interpretability via von-Neumann entanglement entropy analysis

Innovation

• Quantum-inspired neural network with quantum jump operator
• Differentiable module transforms product states into entanglement
• Joint learning of Hamiltonian and Lindblad operators