About the job
The AWS Analytics Engineering (AAE) organization is the analytics backbone of AWS — we build and operate the data platform that powers business decisions across more than 150 AWS services. Every insight surfaced to AWS product leadership, from service adoption trends to revenue drivers, flows through systems our team designs, builds, and maintains. We operate at massive scale — processing petabytes of data daily through thousands of jobs consisting of transformations, reporting queries, ingestions, and infrastructure management scripts. Our engineers work directly with source systems to procure data, convert it into structured formats, build large-scale processing pipelines, design analytical data models, and maintain infrastructure with the highest security and compliance standards.
Responsibilities
Identify limitations and opportunities in data processing tools, drive improvements and innovation, define data processing guidelines, and ensure best practices in all pipelines designed and reviewed.
Define and own data architecture at the team level — ensuring architecture effectively matches business problems and data challenges with security, scalability, and cost effectiveness.
Produce exemplary code — solutions that are easily usable by customers, inventive, secure, easily maintainable, appropriately scalable, and extensible.
Define and own infrastructure architecture at the team level.
Solve complex ambiguous problems — for example, designing cross-domain data models that unify billing, usage, and service telemetry data, or combining multiple datasets to solve problems that couldn't be solved before.
Effectively split project work into parallel tasks that can be performed by themselves and others and reassembled successfully.
Influence related teams' data architecture and software design.
Drive data engineering best practices — Data Discovery, Naming Conventions, Operational Excellence, Data Security.
Proactively fix data architecture deficiencies and propose larger projects which may require the work of other teams.
Participate in on-call rotation and own operational health of data systems — establish monitoring, alarming, runbooks, and SLA tracking.
Qualifications
Minimum
7+ years of data engineering experience
Experience with data modeling, warehousing and building ETL pipelines
Experience with SQL
Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS
Experience mentoring team members on best practices
Experience with MPP databases such as Amazon Redshift
Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets
Preferred
Experience with big data technologies such as: Hadoop, Hive, Spark, EMR
Experience operating large data warehouses
Experience providing technical leadership and mentoring other engineers for best practices on data engineering
Bachelor's degree in computer science, engineering, analytics, mathematics, statistics, IT or equivalent
Knowledge of distributed systems as it pertains to data storage and computing