About the job
We are seeking an experienced systems generalist who can work comfortably across the stack to help build an automated inference optimization platform. Given a workload, target hardware profile, compiler and runtime context, and a trusted verifier, the system runs durable optimization campaigns that generate, compile, execute, grade, and improve candidate kernels, runtime configurations, and serving-stack changes.
You will design both the OpenAI-hosted control plane and the partner-side software that evaluates candidates on real accelerator hardware. The product must keep long-running workflows reliable, make performance results reproducible, and maintain clear trust boundaries around sensitive model and hardware information.
This is a deeply cross-stack role, combining strong software engineering fundamentals with systems thinking and performance intuition. You will work closely with research, inference engineering, infrastructure, security, product, and strategic partners to turn a powerful research workflow into a scalable product.
Responsibilities
- Design, build, and operate durable APIs and control-plane services for multi-hour or multi-day optimization campaigns, including scheduling, retries, budgets, checkpoints, artifact lineage, and observability.
- Build secure partner-side runner and grader software that can compile, execute, verify, and benchmark candidate artifacts on third-party accelerator hardware.
- Integrate hardware profiles, ISA and toolchain context, compilers, runtimes, and inference-serving engines into a repeatable optimization workflow.
- Turn research prototypes into reliable product surfaces with clear contracts, debuggable failure modes, reproducible outputs, and excellent developer ergonomics.
- Develop correctness and performance evaluation systems spanning latency, throughput, memory use, utilization, and cost efficiency.
- Build artifact, provenance, and qualification workflows that make optimized kernels, binaries, configurations, and reports safe to review and deploy.
- Collaborate with Research, Inference Engineering, Infrastructure, Security, Product, and Strategic Partnerships to deliver production-ready solutions.
- Drive technical architecture and execution across ambiguous, cross-functional initiatives that connect OpenAI systems with partner environments.
Qualifications
Minimum
- 8+ years of professional software engineering experience building large-scale distributed systems, infrastructure platforms, or cloud services, or equivalent depth of experience.
- Strong programming skills in one or more of C++, Python, Go, or Rust.
- Experience designing and operating highly available backend systems, APIs, job orchestration systems, or durable workflows for production workloads.
- Strong understanding of distributed systems, Linux, networking, storage, containers, and modern cloud architectures.
- Experience debugging complex systems and using measurement, profiling, and benchmarks to guide engineering decisions.
- Proven ability to lead complex technical initiatives as a senior individual contributor and work effectively across organizational boundaries.
Preferred
- Experience with AI infrastructure, inference-serving systems, or large-scale machine learning systems.
- Experience with compilers, runtimes, kernel optimization, or performance engineering; familiarity with technologies such as LLVM, MLIR, Triton, CUDA, or ROCm is a plus.
- Familiarity with GPUs, accelerators, hardware architecture, ISA concepts, or vendor toolchains.
- Experience with inference-serving frameworks or engines such as vLLM, SGLang, Triton Inference Server, or similar systems.
- Experience building developer platforms, external APIs, remote execution systems, or secure partner-facing infrastructure.
- Experience working with strategic cloud, hardware, or infrastructure partners.