🤖 AI Summary
This work addresses the limitations of traditional numerical array programs, which rely on manual parallelization constrained by static optimizations or explicit annotations, resulting in coarse-grained parallelism and poor adaptability to heterogeneous hardware. The paper proposes a self-optimizing Virtual Processor (VP) that automatically and dynamically parallelizes entire program regions at runtime through a decentralized network of collaborating execution segments, without developer intervention. Its key innovation lies in parallelizing and distributing the scheduling process itself, integrating dependency-driven local decisions, heterogeneity-aware task placement and data movement, and support from the ILNumerics.ONAL instruction set. This approach preserves sequential semantics while enabling automatic parallelism extraction across large-scale program regions, achieving low-latency strong scaling on local heterogeneous systems for a broad range of workloads—from latency-sensitive small operations to large data-parallel tasks—without requiring explicit parallel programming.
📝 Abstract
This work introduces a self-optimizing virtual processor (VP) for numerical array programs that shifts parallelization from a manual developer task to a cooperative, agent-like runtime mechanism. Instead of relying on centralized task-graph scheduling, static compiler optimization, or explicitly annotated parallel constructs, the VP uses a decentralized network of cooperative execution segments, derived from the stream of numerical instructions and their data dependencies at runtime. Each segment makes only local decisions about when, where, and how to prepare and execute its computation, including task placement, kernel preparation, and data movement. No central scheduler or mapper instance determines the execution globally; instead, scheduling itself is parallelized and distributed over time - asynchronously and strictly dependency driven. The overall execution strategy emerges from concurrently executing local segments, continuously responding to data availability, cost estimates, system state, hardware capabilities, and problem size. While preserving the sequential semantics of the program our VP automatically exploits parallelism across large program regions rather than being limited to individual loop bodies, modules, or explicitly marked parallel sections; developers are not required to design or encode a parallelization strategy. The current VP primarily targets low-latency strong scaling on local heterogeneous hardware, covering workloads from small, latency-sensitive array operations to large data-parallel computations. The current implementation targets the predefined array instruction set of the ILNumerics.ONAL domain-specific language, while the underlying concept is applicable to general array-based numerical programming models such as MATLAB and NumPy.