About the job
This position sits within Nebius Token Factory, our serverless platform for running and customizing open-source LLMs in production. Token Factory allows for serverless inference and fine-tuning (LoRA, full FT, RFT) backed by in-house optimizations like custom speculative decoding, quantization, cache-aware routing and dedicated endpoints. Customers come to us to move from prototype to scaled production without the cost and complexity of building and tuning their own inference stack.
Responsibilities
Own the most complex, highest-stakes customer engagements from architecture through production across multiple modalities, driving measurable business value
Optimize LLM inference at the framework and hardware level and codify the resulting best practices into reusable playbooks for the team
Lead supervised and reinforcement fine-tuning efforts to maximize model quality
Design and implement production-ready LLM solutions using Token Factory's inference services
Provide deep technical expertise in prompt engineering, RAG architectures, model selection, and cost/performance trade-offs at scale
Partner closely with product, engineering and research to surface customer needs, prototype platform features, and directly influence the roadmap
Guide customers from PoC to production with a focus on performance, reliability, and cost efficiency — and define the standards by which the team does so
Mentor Senior and mid-level Solutions Architects; raise the technical bar of the team through review, enablement, and knowledge sharing
Represent Token Factory externally through talks, blog posts, and conferences
Qualifications
Minimum
8+ years of experience in ML/AI systems, with at least 4 years focused on LLMs and generative AI
Demonstrated technical leadership: owning ambiguous, high-impact problems end to end and influencing decisions across teams and customers
Expert knowledge of the LLM ecosystem: model architectures, fine-tuning approaches, and inference internals
Deep, hands-on command of inference optimization: quantization, KV-cache management, batching, routing, etc.
Hands-on experience with:
Running LLMs in production at scale: deploying, operating, and debugging inference workloads down to the framework level
LLM fine-tuning, including SFT/LoRA and data preparation/curation; experience with RL-based fine-tuning
LLM evaluation: building task-specific benchmarks and offline/online eval pipelines, including LLM-as-a-judge setups
Inference frameworks and libraries (vLLM, SGLang, TensorRT-LLM), including the ability to read, modify, and contribute to their internals
Deploying LLM-powered applications using APIs from OpenAI, Anthropic, or open-source models
Strong Python programming skills
Excellent communication skills, with the ability to clearly explain technical concepts to diverse audiences, from engineers to executives
Preferred
Contributions or maintainership in major OSS inference/ML projects (vLLM, SGLang, TensorRT-LLM)
Published research, conference talks, or widely-read technical writing in the LLM/serving space
Deep work with multimodal AI models (vision-language, speech)
Proficiency with DevOps tooling (Docker, Kubernetes) and infrastructure-as-code
Experience building or owning internal tooling/automation for ML workflows at scale