About the job
Yahoo Mail is the ultimate consumer inbox with hundreds of millions of users. It's the best way to access your email and stay organized from a computer, phone or tablet. With its beautiful design and lightning fast speed, Yahoo Mail makes reading, organizing, and sending emails easier than ever. As part of Yahoo Mail Data Science & Analytics, our team is the driving force behind experimental analysis, data science, and business intelligence for Yahoo Mail. We thrive on data and insights to craft outstanding product experiences. Our commitment is towards providing data-driven guidance that fuels user growth and product innovation, maintaining high-quality data standards, and devising insightful AB tests and advanced analyses. Mail Intelligence is the intelligent layer of Yahoo Mail, responsible for overseeing end-to-end AI and machine learning components within the platform. The squad delivers cutting-edge AI capabilities that enhance user experience, scales personalization features, powers AI-driven functionalities (including generative AI and LLMs), and provides smart signals and insights driven by Mail data across the Yahoo ecosystem.
Responsibilities
Partner with Mail Intelligence product, engineering, and research science teams to frame AIML challenges into data-analyzable questions, devise analysis strategies, and translate findings into actionable insights.
Design and lead AB testing and experimentation for backend ML features including classification models, extraction pipelines, TLDR summarization, personalization, and ranking/arbitration logic.
Evaluate ML model performance through offline and online metrics, coverage analysis, and quality assessments to inform model iteration and deployment decisions.
Conduct ad-hoc data mining and statistical analysis to understand user behaviors, feature adoption, engagement patterns, and the impact of AI-powered experiences.
Develop and maintain measurement frameworks and dashboards (Tableau, Looker) to track Mail Intelligence KPIs, squad OKRs, and feature health.
Advocate for and enforce high standards in data logging and instrumentation to ensure data integrity and accuracy across backend systems.
Collaborate cross-functionally with frontend squads (Inbox & Organize, Commerce, Read Write Search) to ensure alignment of experimentation logic and data flows for features powered by Mail Intelligence signals.
Drive the development of reporting strategies to accurately reflect business results and guide future AIML investment decisions.
Qualifications
Minimum
Minimum of 7+ years of work experience (minimum of 4 years with a Ph.D.) in data science, product analytics, or a similar role in consumer technology, AIML products, digital media, or product development.
Master's Degree in Math, Engineering, Statistics, Data Science, Computer Science, or a related field.
Subject matter expertise in consumer internet products, ideally with exposure to AIML-powered features.
Strong understanding of experimentation design and analysis in complex, multi-layered systems.
Preferred
Experience working directly with ML/AI engineering and research science teams.
Experience in email, messaging, or communication platforms.
Familiarity with NLP, LLMs, and generative AI measurement and evaluation.
Experience with project management tools like those from Atlassian (JIRA, Confluence).
Experience with GCP (Vertex AI, BigQuery) and cloud-based ML infrastructure.