Neuro-Symbolic Software Verification: Hyper-charging Local Language Models with Symbolic Reasoning at Scale

2026-06-15

AI Summary

This work addresses the bottleneck of loop invariant synthesis in formal verification by introducing VerIbmc, the first fully local neurosymbolic framework that operates without reliance on cloud-based large language model APIs. The approach integrates deterministic symbolic reasoning, locally deployed open-source large language models (ranging from 7B to 120B parameters), the ESBMC model checker, and a structured feedback mechanism, supporting both Chain-of-Thought and Tree-of-Thought prompting strategies. Evaluated on 499 benchmark problems, the best configuration (GPT-OSS-120B) solves 431 instances (86.4%), matching the performance of state-of-the-art cloud-based tools. Notably, the symbolic component alone solves 75 problems and substantially enhances the efficacy of weaker models, achieving efficient verification while preserving code privacy and minimizing computational cost.

Abstract

Loop invariant synthesis remains a central and pivotal bottleneck in formal software verification. Recent LLM-based Neuro-Symbolic tools have achieved impressive solve rates. However, these tools rely on proprietary, often expensive cloud APIs, which constitute a hurdle for privacy-sensitive industrial deployments where the source code cannot leave the organisation or where cost is a factor. We present VerIbmc, a neuro-symbolic pipeline that pairs symbolic invariant generation with locally deployable open-weight language models with the ESBMC verification tool. Our pipeline combines a deterministic symbolic invariant synthesis phase with an iterative LLM refinement loop driven by structured verifier feedback. In addition, we provide two types of pipelines that differ in their prompting strategy: Chain-of-Thought vs. Tree-of-Thought. We conduct an extensive experimental evaluation with five open-weight models (ranging from 7B to 120B parameters) across five benchmark families comprising of 520 problems (499 after excluding 21 with unavoidable overflow). Overall, the best single configuration (GPT-OSS-120B) solves 431 of 499 problems (86.4%). Additionally, on the four benchmark suites shared with the strongest cloud-API tools, VerIbmc is competitive running only on a single local machine. The evaluation shows symbolic invariant synthesis solves 75 problems without any LLM call and yields up to +35 additional problems for the weakest model. Importantly, all inference runs entirely on a single local machine using open-weight models -- no cloud API or proprietary model is required. Overall, we demonstrate that a neuro-symbolic approach based on LLMs can be used effectively for invariant synthesis in a privacy-preserving and energy-efficient manner, without having to resort to expensive proprietary frontier models locked behind APIs.

Problem

• loop invariant synthesis
• formal software verification
• privacy-sensitive deployment
• cloud API dependency
• neuro-symbolic reasoning

Innovation

• Neuro-Symbolic Verification
• Loop Invariant Synthesis
• Local Language Models
• Symbolic Reasoning
• Privacy-Preserving Verification