Trim My View: An LLM-Based Code Query System for Module Retrieval in Robotic Firmware

2025-03-05

AI Summary

Identifying module functionality and semantics in stripped robotic firmware binaries remains challenging due to the absence of symbolic information. Method: This paper proposes a lightweight module retrieval system that first reconstructs function structures via static analysis and multi-pass decompilation, then leverages open-source code large language models (CodeStral, DeepSeek-Coder, CodeQwen) to generate natural-language semantic summaries for each function, thereby constructing an interpretable cross-modal (code ↔ text) module index. Contribution/Results: To our knowledge, this is the first work to systematically integrate code LLMs into firmware reverse-engineering for module identification, enabling end-to-end semantic mapping without symbols. Evaluated on 467 modules across four ArduPilot platforms, the best-performing model—CodeStral—achieves an average F1-score of 0.68, with query latency of only several seconds per retrieval.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Image and Video Retrieval

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The software compilation process has a tendency to obscure the original design of the system and makes it difficult both to identify individual components and discern their purpose simply by examining the resulting binary code. Although decompilation techniques attempt to recover higher-level source code from the machine code in question, they are not fully able to restore the semantics of the original functions. Furthermore, binaries are often stripped of metadata, and this makes it challenging to reverse engineer complex binary software. In this paper we show how a combination of binary decomposition techniques, decompilation passes, and LLM-powered function summarization can be used to build an economical engine to identify modules in stripped binaries and associate them with high-level natural language descriptions. We instantiated this technique with three underlying open-source LLMs -- CodeQwen, DeepSeek-Coder and CodeStral -- and measured its effectiveness in identifying modules in robotics firmware. This experimental evaluation involved 467 modules from four devices from the ArduPilot software suite, and showed that CodeStral, the best-performing backend LLM, achieves an average F1-score of 0.68 with an online running time of just a handful of seconds.

Problem

• Identify modules in stripped binaries using LLM-based techniques.
• Associate binary modules with high-level natural language descriptions.
• Evaluate effectiveness in robotics firmware with multiple LLMs.

Innovation

• Combines binary decomposition and decompilation techniques
• Uses LLM-powered function summarization for module identification
• Evaluates with open-source LLMs on robotics firmware