Senior Software Engineer, ML Infrastructure

Decagon
San Francisco2026-03-26OnSite

About the job

We're hiring a Senior ML Infrastructure Engineer to own the platforms powering Decagon's model training and inference. You'll build distributed training systems, design inference architecture across multiple providers, and create the frameworks that let our Research and Product teams ship faster. This role is for someone who thrives on technical depth, can lead multi-quarter initiatives, and wants to shape the long-term architecture of our ML stack.

Responsibilities

Design and build distributed training platforms for LLM and multimodal fine-tuning and post-training at scale

Integrate state-of-the-art training algorithms into production pipelines

Own inference architecture and multi-provider routing, including failover and optimization

Lead initiatives to improve latency and cost efficiency across the training and serving stack

Build evaluation and experimentation infrastructure that enables rapid, reliable iteration

Drive technical direction, mentor engineers, and establish best practices for ML infrastructure

Qualifications

Minimum

6+ years building ML infrastructure or production systems at scale

Deep experience with distributed training: multi-node GPU clusters, fault tolerance, and optimization

Strong understanding of LLM inference: latency optimization, provider tradeoffs, and serving architecture

Proven track record leading complex, multi-quarter technical projects

Preferred

No preferred qualifications listed.