About the job
NVIDIA is looking for a highly technical Product Manager to own the products that help customers extract the best possible performance from AI models and applications running on NVIDIA hardware. Every inference deployment — from a single-GPU workstation to a multi-thousand-GPU data center — lives or dies on latency, efficiency, and cost per token. Your job is to make NVIDIA the obvious place to run inference by turning deep optimization techniques into products that a broad range of customers can actually adopt. The work spans the entire inference stack — optimization techniques, the frameworks that deliver them; and the benchmarking and operational tooling customers rely on to trust the results. You will translate what our best-performing internal deployments do into capabilities that ship, are detailed, and work for everyone else. The Product Management organization at NVIDIA is a small, high-leverage team driving the company’s Deep Learning and Generative AI strategy. We need a self-starter who can operate with minimal direction, form a point of view from data and customer conversations, and drive it to a shipped result.
Responsibilities
Own the inference performance roadmap and set direction across the stack including model representation, memory management, scheduling, and token generation.
Build platforms that generalize across model families, deployment topologies, and customer sizes with easy adoption and extensibility.
Define the performance strategy for agentic applications and multi-turn workloads, driving capabilities around cross-turn cache reuse, request prioritization, and efficient handling of idle time.
Define how optimizations land across TensorRT-LLM, vLLM, SGLang, and NVIDIA Dynamo by partnering with open-source communities and internal engineering teams.
Own benchmarking methodology and performance claims, defining metrics like TTFT, ITL, throughput per GPU, and cost per million tokens.
Run the product day to day including release readiness, quality bars, regression tracking, customer blocking issues, and feedback loops from production deployments.
Qualifications
Minimum
12+ years in product management at a technology company, or comparable time as a founder, engineering lead, or technical product owner.
Depth in AI inference optimization: KV caching and reuse, quantization, speculative decoding, disaggregated serving.
Familiarity with the inference and orchestration frameworks customers use: TensorRT-LLM, vLLM, SGLang, NVIDIA Dynamo, and the surrounding serving and scheduling ecosystem.
Proven track record of working independently to take an ambiguous problem space, define the strategy, and drive it to a shipped outcome.
Operational experience running a live product: release management, quality and regression rigor, customer issues, and support processes.
Skill at translating low-level capability into business value — lower TCO, faster response, better GPU utilization — for engineers and executives alike.
BS, MS, or PhD in Computer Science, Computer Engineering, or another relevant area of study (or equivalent experience).
Preferred
Engineering experience with LLM inference performance: profiling, kernel-level or serving-level optimization, or building a serving stack.
Open-source contributions or product leadership in vLLM, SGLang, TensorRT-LLM, Triton Inference Server, or Dynamo.
Production experience at scale: capacity planning, autoscaling, SLA management, or stateful multi-turn applications.
A habit of reading the relevant research and translating it into roadmap decisions with intuition for where model architectures and serving techniques are heading next.