KEENHash: Hashing Programs into Function-Aware Embeddings for Large-Scale Binary Code Similarity Analysis

2025-06-13

AI Summary

To address the poor scalability of function-level matching in large-scale binary code similarity analysis (BCSA), this paper proposes KEENHash—the first end-to-end hashing framework designed for program-level similarity retrieval. KEENHash jointly leverages LLM-generated function-level semantic embeddings, K-Means–guided locality-sensitive hashing, and feature compression to produce fixed-length, function-aware program-level hash codes. Unlike conventional per-function matching approaches, KEENHash reduces time complexity to sublinear, enabling real-time program similarity retrieval at billion-scale. Evaluated on 5.3 billion similarity comparisons, KEENHash completes inference in just 395.83 seconds—accelerating over state-of-the-art methods by at least 215×—while improving malware detection accuracy by ≥23.16%.

Technology Category

• Machine Learning: Hardware-aware ML
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Binary code similarity analysis (BCSA) is a crucial research area in many fields such as cybersecurity. Specifically, function-level diffing tools are the most widely used in BCSA: they perform function matching one by one for evaluating the similarity between binary programs. However, such methods need a high time complexity, making them unscalable in large-scale scenarios (e.g., 1/n-to-n search). Towards effective and efficient program-level BCSA, we propose KEENHash, a novel hashing approach that hashes binaries into program-level representations through large language model (LLM)-generated function embeddings. KEENHash condenses a binary into one compact and fixed-length program embedding using K-Means and Feature Hashing, allowing us to do effective and efficient large-scale program-level BCSA, surpassing the previous state-of-the-art methods. The experimental results show that KEENHash is at least 215 times faster than the state-of-the-art function matching tools while maintaining effectiveness. Furthermore, in a large-scale scenario with 5.3 billion similarity evaluations, KEENHash takes only 395.83 seconds while these tools will cost at least 56 days. We also evaluate KEENHash on the program clone search of large-scale BCSA across extensive datasets in 202,305 binaries. Compared with 4 state-of-the-art methods, KEENHash outperforms all of them by at least 23.16%, and displays remarkable superiority over them in the large-scale BCSA security scenario of malware detection.

Problem

• Efficient large-scale binary code similarity analysis
• Reducing time complexity in function-level diffing
• Improving accuracy in program clone search

Innovation

• LLM-generated function embeddings for hashing
• K-Means and Feature Hashing for compact embeddings
• Efficient large-scale binary similarity analysis