About the job
The Annapurna Labs team at Amazon Web Services (AWS) builds AWS Neuron, the software development kit used to accelerate deep learning and GenAI workloads on Amazon’s custom machine learning accelerators, Inferentia and Trainium. This role is responsible for development, enablement and performance tuning of a wide variety of LLM model families, including massive scale large language models like the Llama family, DeepSeek and beyond. The Inference Enablement and Acceleration team works side by side with compiler engineers and runtime engineers to create, build and tune distributed inference solutions with Trainium and Inferentia.
Responsibilities
Design, develop, and optimize machine learning models and frameworks for deployment on custom ML hardware accelerators.
Participate in all stages of the ML system development lifecycle including distributed computing based architecture design, implementation, performance profiling, hardware-specific optimizations, testing and production deployment.
Build infrastructure to systematically analyze and onboard multiple models with diverse architecture.
Design and implement high-performance kernels and features for ML operations, leveraging the Neuron architecture and programming models
Analyze and optimize system-level performance across multiple generations of Neuron hardware
Conduct detailed performance analysis using profiling tools to identify and resolve bottlenecks
Implement optimizations such as fusion, sharding, tiling, and scheduling
Conduct comprehensive testing, including unit and end-to-end model testing with continuous deployment and releases through pipelines.
Work directly with customers to enable and optimize their ML models on AWS accelerators
Collaborate across teams to develop innovative optimization techniques
Qualifications
Minimum
Experience optimizing inference performance for both latency and throughput on such large models across the stack from system level optimizations through to Pytorch or JAX is a must have.
Strong software development using Python, System level programming and ML knowledge are both critical to this role.
Preferred
You will architect and implement business critical features, and mentor a brilliant team of experienced engineers.
Working at the intersection of software, hardware, and machine learning systems, you'll bring expertise in low-level optimization, system architecture, and ML model acceleration.