About the job
The Annapurna Labs team at Amazon Web Services (AWS) builds AWS Neuron, the software development kit used to accelerate deep learning and GenAI workloads on Amazon’s custom machine learning accelerators, Inferentia and Trainium. The Inference Enablement and Acceleration team is at the forefront of running a wide range of models and supporting novel architecture alongside maximizing their performance for AWS's custom ML accelerators. This role offers a unique opportunity to work at the intersection of machine learning, high-performance computing, and distributed architectures, where you'll help shape the future of AI acceleration technology.
Responsibilities
would with state of the art LLMs, Open source and internal LLM families, large scale performance and benchmark evaluations etc.,
develop and performance tune a wide variety of LLM model families, including 500B+ large language models like the Llama family, DeepSeek and beyond.
work side by side with performance, compiler and runtime engineers to create, build and tune distributed inference solutions with Trainium and Inferentia.
build infrastructure to systematically analyze and onboard multiple models with diverse architecture.
collaborate with performance team to enable and evaluate optimizations such as fusion, sharding, tiling, and scheduling etc.,
conduct comprehensive testing, including unit and end-to-end model testing with continuous deployment and releases through pipelines.
work directly with customers to enable and optimize their ML models on AWS accelerators
collaborate across teams to develop innovative optimization techniques
Build online/offline inference serving with vLLM, SGLang, TensorRT or similar platforms in production environments.
Qualifications
Minimum
Experience optimizing inference performance for both latency and throughput on such large models across the stack from system level optimizations through to Pytorch or JAX is a must have.
Strong software development using Python, System level programming and ML knowledge are both critical to this role.
Preferred
You will architect and implement business critical features, and mentor a brilliant team of experienced engineers.
Our engineers collaborate across compiler, runtime, framework, and hardware teams to optimize machine learning workloads for our global customer base.
Working at the intersection of software, hardware, and machine learning systems, you'll bring expertise in low-level optimization, system architecture, and ML model acceleration.