PQFA: Parallel Quantum Feature Augmentation of Fused Representations for Multimodal Classification

2026-07-15

AI Summary

This work addresses the limitations of insufficient feature enhancement after fusion and limited robustness under missing modalities in multimodal learning by proposing a hybrid quantum-classical framework. Building upon frozen RoBERTa and ViT encoders, the method generates joint representations through bidirectional cross-attention, attention pooling, and adaptive gating, then amplitude-encodes these representations into multiple parallel shallow variational quantum circuits for efficient feature enhancement. The resulting quantum measurement outcomes are concatenated with classical representations for final classification. This approach is the first to integrate parallel variational quantum circuits into multimodal feature enhancement, significantly outperforming non-enhanced baselines and parameter-comparable MLPs—achieving superior performance with only ~2.2K parameters compared to 24.0K for MLPs—on MM-IMDb and N24News benchmarks, while also demonstrating enhanced robustness when the text modality is severely degraded.

Abstract

Most multimodal learning methods improve how heterogeneous representations are aligned and fused, while post-fusion enhancement remains less explored. We propose Parallel Quantum Feature Augmentation (PQFA), a hybrid quantum-classical framework that applies multiple shallow variational quantum circuits to fused multimodal features. Text and image representations extracted by frozen RoBERTa and ViT encoders are processed through bidirectional cross-attention, attentive pooling, and adaptive gated fusion. The fused feature is then amplitude-encoded into parallel quantum circuits, whose measurement readouts are concatenated with the classical representation for prediction. We evaluate PQFA on MM-IMDb and N24News through controlled comparisons using the same encoders, fusion backbone, data splits, projection dimension, and augmentation output width. PQFA consistently outperforms both the fusion backbone without quantum augmentation and a width-matched MLP augmentation baseline, while using approximately 2.2K augmentation parameters compared with 24.0K for the MLP branch. Missing-modality experiments further show improved robustness when textual or visual inputs are incomplete, with particularly clear gains when the more informative textual modality is severely degraded. Controlled ablations and feature-space analyses indicate that the improvement cannot be reproduced by random feature mappings, increased classical width, or untrained quantum transformations. Quantum-state diagnostics additionally show stable predictive performance across the tested simulated noise levels and distinct branch-specific transformations of the encoded states. These results establish PQFA as an effective and parameter-efficient strategy for post-fusion augmentation in hybrid quantum-classical multimodal learning.

Problem

• multimodal classification
• post-fusion enhancement
• feature augmentation
• quantum-classical learning
• fused representations

Innovation

• Quantum Feature Augmentation
• Multimodal Fusion
• Variational Quantum Circuits
• Hybrid Quantum-Classical Learning
• Post-fusion Enhancement