Abduction of Domain Relationships from Data for VQA

2025-02-11

Electronic Proceedings in Theoretical Computer Science

AI Summary

This paper addresses the challenge in visual question answering (VQA) where both images and questions are represented as Answer Set Programming (ASP) programs, and domain-specific annotated data are scarce. To tackle this, we propose a logic abduction-based approach for modeling domain relations—requiring no external knowledge base and instead automatically inferring implicit semantic relationships among image constituents from a small set of historical examples. This enables the induction and learning of transferable logical rules. Our key contribution is the first application of logical abduction to domain knowledge acquisition in VQA, integrating ASP formalization, example-driven relation induction, and lightweight rule learning. Experiments under low-shot settings demonstrate significant improvements in VQA accuracy, validating the method’s effectiveness, generalizability, and practicality in scenarios with limited or no domain annotations.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Computer Vision: Visual Reasoning & Symbolic Representations
• Machine Learning: Statistical Relational/Logic Learning

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Web Mining and Content Analysis: Community question answering
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

In this paper, we study the problem of visual question answering (VQA) where the image and query are represented by ASP programs that lack domain data. We provide an approach that is orthogonal and complementary to existing knowledge augmentation techniques where we abduce domain relationships of image constructs from past examples. After framing the abduction problem, we provide a baseline approach, and an implementation that significantly improves the accuracy of query answering yet requires few examples.

Problem

• Abduction of domain relationships
• Improving VQA accuracy
• Minimal example requirement

Innovation

• Abduction of domain relationships
• Orthogonal to knowledge augmentation
• Improves query answering accuracy