Hardware-efficient tractable probabilistic inference for TinyML Neurosymbolic AI applications

2025-07-07

AI Summary

Symbolic models in neural-symbolic AI (NSAI) for TinyML devices suffer from limited scalability due to mismatched sparsity and low-precision neural computation. Method: We propose a hardware-software co-design framework featuring a hardware-friendly, deterministic probabilistic circuit (PC) training method, incorporating traceable PC architectures and an *n*-th root compression algorithm to preserve symbolic reasoning accuracy under extreme resource constraints. The solution is deployed on FPGA and ESP32 platforms. Contribution/Results: Compared to a 64-bit baseline, our implementation reduces FPGA logic resources (FFs/LUTs) by 82.3% and 52.6%, respectively, and cuts Flash usage by 18.0%; on ESP32, it achieves 4.67× average inference speedup. This work presents the first high-accuracy, compact, and interpretable symbolic model deployment on ultra-low-power edge devices—establishing a new paradigm for trustworthy edge AI.

Technology Category

• Machine Learning: Hardware-aware ML
• Cognitive Modeling & Cognitive Systems: Symbolic Representations
• Computer Vision: Visual Reasoning & Symbolic Representations

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Security and Privacy: Security and privacy of machine learning and AI applications

Abstract

Neurosymbolic AI (NSAI) has recently emerged to mitigate limitations associated with deep learning (DL) models, e.g. quantifying their uncertainty or reason with explicit rules. Hence, TinyML hardware will need to support these symbolic models to bring NSAI to embedded scenarios. Yet, although symbolic models are typically compact, their sparsity and computation resolution contrasts with low-resolution and dense neuro models, which is a challenge on resource-constrained TinyML hardware severely limiting the size of symbolic models that can be computed. In this work, we remove this bottleneck leveraging a tight hardware/software integration to present a complete framework to compute NSAI with TinyML hardware. We focus on symbolic models realized with tractable probabilistic circuits (PCs), a popular subclass of probabilistic models for hardware integration. This framework: (1) trains a specific class of hardware-efficient emph{deterministic} PCs, chosen for the symbolic task; (2) emph{compresses} this PC until it can be computed on TinyML hardware with minimal accuracy degradation, using our $n^{th}$-root compression technique, and (3) emph{deploys} the complete NSAI model on TinyML hardware. Compared to a 64b precision baseline necessary for the PC without compression, our workflow leads to significant hardware reduction on FPGA (up to 82.3% in FF, 52.6% in LUTs, and 18.0% in Flash usage) and an average inference speedup of 4.67x on ESP32 microcontroller.

Problem

• Enable efficient probabilistic inference for TinyML Neurosymbolic AI
• Overcome hardware limitations for symbolic models in embedded scenarios
• Compress and deploy deterministic PCs on resource-constrained TinyML hardware

Innovation

• Hardware-efficient deterministic probabilistic circuits
• Nth-root compression technique for models
• Tight hardware/software integration framework