Chart-RL: Policy Optimization Reinforcement Learning for Enhanced Visual Reasoning in Chart Question Answering with Vision Language Models

2026-04-03

AI Summary

Existing vision-language models often underperform on chart question answering due to inaccurate numerical extraction, insufficient understanding of implicit visual relationships, and weak spatial attention mechanisms. This work proposes a training framework that integrates strategic optimization via reinforcement learning with an adaptive reward function, enabling parameter-efficient fine-tuning on a single GPU through LoRA. The approach substantially enhances the model’s perceptual and logical reasoning capabilities on charts. When applied to the Qwen3-VL-4B-Instruct model (4B parameters), it achieves an accuracy of 0.634 on the ChartQAPro benchmark—surpassing the 8B baseline (0.580)—while reducing inference latency from 31 seconds to 9 seconds.

Technology Category

• Computer Vision: Large Vision Models
• Natural Language Processing: Question Answering
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

The recent advancements in Vision Language Models (VLMs) have demonstrated progress toward true intelligence requiring robust reasoning capabilities. Beyond pattern recognition, linguistic reasoning must integrate with visual comprehension, particularly for Chart Question Answering (CQA) tasks involving complex data visualizations. Current VLMs face significant limitations in CQA, including imprecise numerical extraction, difficulty interpreting implicit visual relationships, and inadequate attention mechanisms for capturing spatial relationships in charts. In this work, we address these challenges by presenting Chart-RL, a novel reinforcement learning framework that enhances VLMs chart understanding through feedback-driven policy optimization of visual perception and logical inference. Our key innovation includes a comprehensive framework integrating Reinforcement Learning (RL) from Policy Optimization techniques along with adaptive reward functions, that demonstrates superior performance compared to baseline foundation models and competitive results against larger state-of-the-art architectures. We also integrated Parameter-Efficient Fine-Tuning through Low-Rank Adaptation (LoRA) in the RL framework that only requires single GPU configurations while preserving performance integrity. We conducted extensive benchmarking across open-source, proprietary, and state-of-the-art closed-source models utilizing the ChartQAPro dataset. The RL fine-tuned Qwen3-VL-4B-Instruct model achieved an answer accuracy of 0.634, surpassing the 0.580 accuracy of the Qwen3-VL-8B-Instruct foundation model despite utilizing half the parameter count, while simultaneously reducing inference latency from 31 seconds to 9 seconds.

Problem

• Chart Question Answering
• Vision Language Models
• Visual Reasoning
• Numerical Extraction
• Spatial Relationships

Innovation

• Reinforcement Learning
• Policy Optimization
• Vision Language Models
• Chart Question Answering
• LoRA