Carnot: Interpretable, Interactive, and Optimized Execution of Deep Research Queries

2026-08-10

AI Summary

Existing AI-powered data lake query tools struggle to balance interpretability, user control, and enterprise-grade cost efficiency, hindering analysts’ ability to verify, intervene in, or execute analytical tasks effectively. This work proposes a novel approach that compiles natural language queries into interactive physical execution graphs, integrated within a notebook interface to enable users to inspect execution plans, run computations step-by-step, examine intermediate results, and dynamically edit either code or semantic instructions. Coupled with a dynamic optimization mechanism sensitive to cost and latency constraints, this framework uniquely unifies interpretable execution, user intervention, and resource-aware optimization. Evaluated in real-world enterprise settings, the system substantially enhances analytical controllability, verifiability, and resource efficiency.

Abstract

Enterprises increasingly seek to query data lakes using natural language via AI-driven tools like semantic operators or deep research agents. However, the latter operates as an opaque black box, hiding its intermediate reasoning and data retrieval steps, and failing to expose controls for managing API costs and execution latency. Meanwhile, the former can be prohibitively expensive for enterprise-scale data lakes. Consequently, analysts using these systems lack the agency to intercept hallucinated premises, verify intermediate results, or correct the system's trajectory. We present Carnot, an interactive execution engine for AI-driven analytics. Carnot compiles natural language requests into physical execution graphs and surfaces them through an interactive notebook interface. Rather than waiting blindly for a final output, users can critique the plan, incrementally execute operators, inspect intermediate data, or directly edit the underlying code or semantic operator instructions. Carnot's query optimizer will optimize the query with respect to cost or latency constraints provided by the user. Our demo will showcase how Carnot helps users achieve efficient and verifiable insights on workloads motivated by real enterprise use cases.

Problem

• natural language querying
• data lakes
• black-box AI systems
• cost control
• execution transparency

Innovation

• interpretable AI
• interactive execution
• query optimization
• natural language querying
• data lake analytics