From Interpretation to Compilation: Compilation-Based Execution of Semantic Operators [Vision]

2026-07-14

AI Summary

This work addresses the high latency, cost, and limited scalability of existing semantic operator systems that rely on interpreted execution with frequent large language model (LLM) invocations. The authors propose a compiler-based approach to semantic operator execution, treating the LLM as a semantic compiler that, during a one-time compilation phase, translates its outputs into deterministic, executable code—thereby eliminating the need for repeated LLM calls at runtime. By integrating techniques from database query optimization, program synthesis, and LLM-driven data processing, the method substantially reduces both execution time and the number of LLM invocations while preserving high output quality. The approach has been successfully integrated into an existing system, demonstrating its practical viability.

Abstract

Semantic operator systems extend data processing with natural-language interfaces, supporting operations such as semantic filtering, mapping, and joining. Existing systems commonly execute these operators through interpretation-based execution: for each row, record, or candidate pair, an LLM is invoked to interpret the semantic intent and produce an output. Although expressive, this places expensive LLM calls inside the data-processing loop, causing high latency, monetary cost, and limited scalability. We propose compilation-based execution of semantic operators. Instead of using an LLM as a runtime interpreter for every data item, we invoke it once during compilation to translate a semantic operator specification into deterministic executable code. The generated code serves as a compiled physical operator that approximates the behavior of the original LLM-based operator and runs locally over the dataset without per-row or per-pair LLM calls. We instantiate this approach for semantic filter, semantic map, and semantic join operators, compare it with LLM-based interpreted execution, and integrate it into an existing semantic operator system. Preliminary results show that compilation-based execution substantially reduces execution time and LLM calls while preserving much of the output quality. More broadly, we argue that semantic operator systems should treat LLMs not only as runtime executors, but also as semantic compilers that generate efficient executable plans, opening new research opportunities at the intersection of database query processing, program synthesis, and LLM-powered data systems.

Problem

• semantic operators
• interpretation-based execution
• large language models
• data processing latency
• scalability

Innovation

• compilation-based execution
• semantic operators
• large language models
• program synthesis
• query processing