Man-Made Heuristics Are Dead. Long Live Code Generators!

2025-10-09

AI Summary

Traditional system controllers rely on expert-crafted heuristic policies, suffering from limited generalizability and adaptability. This paper introduces PolicySmith, the first framework to automate the synthesis of system-level heuristics using large language models (LLMs). It integrates prompt engineering, feedback-driven iterative code generation, and domain-specific system modeling to search for instance-optimal policies tailored to concrete deployment scenarios. The approach addresses canonical problems including web caching and congestion control: on standard cache traces, synthesized policies significantly outperform established baselines; the generated congestion control policies are formally verified for safety and feasibility, and successfully deployed in the Linux kernel. This work establishes a paradigm shift—from manual policy design to AI-native policy generation—enabling cross-domain policy synthesis and direct integration into production systems.

Technology Category

• Search and Optimization: Heuristic Search
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Multiagent Systems: Multiagent Learning

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Policy design for various systems controllers has conventionally been a manual process, with domain experts carefully tailoring heuristics for the specific instance in which the policy will be deployed. In this paper, we re-imagine policy design via a novel automated search technique fueled by recent advances in generative models, specifically Large Language Model (LLM)-driven code generation. We outline the design and implementation of PolicySmith, a framework that applies LLMs to synthesize instance-optimal heuristics. We apply PolicySmith to two long-standing systems policies - web caching and congestion control, highlighting the opportunities unraveled by this LLM-driven heuristic search. For caching, PolicySmith discovers heuristics that outperform established baselines on standard open-source traces. For congestion control, we show that PolicySmith can generate safe policies that integrate directly into the Linux kernel.

Problem

• Automating policy design through generative models for system controllers
• Synthesizing instance-optimal heuristics using LLM-driven code generation
• Generating safe congestion control policies integrated into Linux kernel

Innovation

• Automated policy design using generative models
• LLM-driven code generation for optimal heuristics
• Framework synthesizes policies for caching and congestion