🤖 AI Summary
Existing VideoQA methods suffer from inaccurate and insufficient modeling of static relationships, limiting fine-grained spatiotemporal reasoning. To address this, we propose a dual-graph collaborative modeling framework: (1) an intra-type dual graph enabling fine-grained message passing among objects and relations of the same category; and (2) an inter-type heterogeneous graph explicitly modeling cross-category static relational interactions. We introduce the first static-relationship-driven bidirectional message-passing inference paradigm, integrating graph neural networks, multi-granularity graph updates, and static relationship embedding. Our approach achieves significant accuracy improvements on ANetQA and Next-QA, demonstrating that static-relationship-guided joint graph reasoning substantially enhances video understanding.
📝 Abstract
Video Question Answering (VideoQA) is an important research direction in the field of artificial intelligence, enabling machines to understand video content and perform reasoning and answering based on natural language questions. Although methods based on static relationship reasoning have made certain progress, there are still deficiencies in the accuracy of static relationship recognition and representation, and they have not fully utilized the static relationship information in videos for in-depth reasoning and analysis. Therefore, this paper proposes a reasoning method for intra-type and inter-type message passing based on static relationships. This method constructs a dual graph for intra-type message passing reasoning and builds a heterogeneous graph based on static relationships for inter-type message passing reasoning. The intra-type message passing reasoning model captures the neighborhood information of targets and relationships related to the question in the dual graph, updating the dual graph to obtain intra-type clues for answering the question. The inter-type message passing reasoning model captures the neighborhood information of targets and relationships from different categories related to the question in the heterogeneous graph, updating the heterogeneous graph to obtain inter-type clues for answering the question. Finally, the answers are inferred by combining the intra-type and inter-type clues based on static relationships. Experimental results on the ANetQA and Next-QA datasets demonstrate the effectiveness of this method.