About the job
The Annapurna Labs team at Amazon Web Services (AWS) builds AWS Neuron, the software development kit used to accelerate deep learning and GenAI workloads on Amazon’s custom machine learning accelerators, Inferentia and Trainium. The Inference Enablement and Acceleration team is at the forefront of running a wide range of models and supporting novel architecture alongside maximizing their performance for AWS's custom ML accelerators.
Responsibilities
Design, develop, and optimize machine learning models and frameworks for deployment on custom ML hardware accelerators.
Participate in all stages of the ML system development lifecycle including distributed computing based architecture design, implementation, performance profiling, hardware-specific optimizations, testing and production deployment.
Build infrastructure to systematically analyze and onboard multiple models with diverse architecture.
Design and implement high-performance kernels and features for ML operations, leveraging the Neuron architecture and programming models
Analyze and optimize system-level performance across multiple generations of Neuron hardware
Conduct detailed performance analysis using profiling tools to identify and resolve bottlenecks
Implement optimizations such as fusion, sharding, tiling, and scheduling
Conduct comprehensive testing, including unit and end-to-end model testing with continuous deployment and releases through pipelines.
Work directly with customers to enable and optimize their ML models on AWS accelerators
Collaborate across teams to develop innovative optimization techniques
Qualifications
Minimum
Experience optimizing inference performance for both latency and throughput on such large models across the stack from system level optimizations through to Pytorch or JAX is a must have.
Strong software development using Python, System level programming and ML knowledge are both critical to this role.
Preferred
Experience optimizing inference performance for both latency and throughput on such large models across the stack from system level optimizations through to Pytorch or JAX is a must have.