Self-Evolving Neuro-Symbolic Skills for Tool-Augmented Spatial Reasoning

2026-08-08

AI Summary

This work addresses the unreliability of large vision-language models in fine-grained spatial reasoning, stemming from their limited capacity for precise spatial perception and geometric computation. To overcome this limitation, we propose NeSy-Spatial, a novel framework that introduces, for the first time, a self-evolving neuro-symbolic skill mechanism. This mechanism abstracts tool invocation and geometric operations into executable atomic instructions, enabling dynamic composition, optimization, and reuse of skills through closed-loop reasoning and trajectory replay. By integrating neuro-symbolic systems, tool-augmented learning, and a skill retrieval-execution pipeline, NeSy-Spatial achieves substantial accuracy gains across three spatial reasoning benchmarks, demonstrating more precise tool usage and stronger cross-task generalization.

Abstract

Large vision-language models have achieved strong performance in multimodal reasoning, but they remain unreliable on fine-grained spatial tasks that demand both precise spatial perception and fine-grained geometric computation beyond end-to-end generation. Tool augmentation offers a natural solution, while existing methods either plan tool calls from scratch without explicit dependency constraints or rely on fixed pipelines that are redundant and generalize poorly across spatial tasks. An effective spatial reasoning agent should instead accumulate reusable experience and adaptively compose it for new problems. To this end, we propose NeSy-Spatial, a neuro-symbolic framework for self-evolving spatial skills. NeSy-Spatial abstracts tool interactions and geometric operations into typed executable atomic instructions and composes them into two complementary skill types: Tool-Use Skills for organizing tool execution and Geometry Skills for structured geometric reasoning. During inference, NeSy-Spatial retrieves and executes relevant skills in a closed-loop process. During evolution, it analyzes buffered successful and failed trajectories to refine skill structures and prune unreliable or inactive entries. Experiments on three spatial reasoning benchmarks show that NeSy-Spatial consistently improves reasoning accuracy with more precise tool utilization.

Problem

• spatial reasoning
• tool augmentation
• fine-grained geometric computation
• multimodal reasoning
• generalization

Innovation

• neuro-symbolic
• self-evolving skills
• tool-augmented reasoning
• spatial reasoning
• atomic instructions