Iridescent: A Framework Enabling Online System Implementation Specialization

2025-08-21

AI Summary

Achieving efficient and general-purpose performance optimization remains challenging for systems operating under dynamic workloads and heterogeneous hardware. Method: This paper proposes an online automatic specialization framework that integrates runtime performance feedback with just-in-time (JIT) compilation to dynamically explore developer-defined specialization spaces—generating optimal specialized configurations without source-code modification or compromising generality. Contribution/Results: Its key innovation is the first integration of a lightweight online search strategy into the JIT pipeline, enabling low-overhead, continuously evolving specialization decisions. The framework supports cross-platform, multi-granularity (function- and module-level) specialization while avoiding the inflexibility inherent in traditional offline specialization. Experimental evaluation on real-world systems demonstrates an average performance improvement of 23.6%, while preserving interface compatibility and developer friendliness.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Machine Learning: Hardware-aware ML

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Specializing systems to specifics of the workload they serve and platform they are running on often significantly improves performance. However, specializing systems is difficult in practice because of compounding challenges: i) complexity for the developers to determine and implement optimal specialization; ii) inherent loss of generality of the resulting implementation, and iii) difficulty in identifying and implementing a single optimal specialized configuration for the messy reality of modern systems. To address this, we introduce Iridescent, a framework for automated online system specialization guided by observed overall system performance. Iridescent lets developers specify a space of possible specialization choices, and then at runtime generates and runs different specialization choices through JIT compilation as the system runs. By using overall system performance metrics to guide this search, developers can use Iridescent to find optimal system specializations for the hardware and workload conditions at a given time. We demonstrate feasibility, effectivity, and ease of use.

Problem

• Automates online system specialization for performance optimization
• Addresses complexity in determining optimal specialization choices
• Enables runtime generation of specialized configurations via JIT compilation

Innovation

• Automated online system specialization framework
• JIT compilation for runtime specialization generation
• Performance metrics guided optimal specialization search