About the job
We're looking for a Senior AI Product Manager who will drive Copilot model capabilities such as tool-use to ensure that the language models that power Microsoft Copilot deliver high quality responses to our users whilst being grounded, reliable, and cost-efficient. You will work at the nexus of product and research, driving execution in partnership with engineers, language engineers, data scientists and researchers. We’re looking for someone with an abundance of positive energy, empathy, and kindness, in addition to being highly effective. The right candidate takes the initiative and enjoys building world-class consumer experiences and products in a fast-paced environment.
Responsibilities
Develop and execute on LLM platform strategy for Copilot that extend language model's capabilities.
Prototype approaches by steering language models to drive response quality across a wide range of scenarios.
Identify and prioritize platform, orchestration and language model issues that impact quality, factuality and safety and working with engineers and researchers to find a path to resolution.
Define and build measurable evaluations with relevant datasets to demonstrate quality improvements.
Define, deploy and manage experiments in production that impact language model's tool use, driving measurable improvements in relevance for and engagement with Copilot users.
Partner with product teams to scale tool building and work with inference, agents and orchestration teams to resolve dependencies.
Accountable to own the status of key projects, proactively identifying risks and proposing solutions to ensure timely delivery.
Qualifications
Minimum
Bachelor's Degree AND 5+ years experience in product management OR equivalent experience.
3+ years of experience leading ambiguous product areas, defining requirements, developing roadmaps, and working with multi-disciplinary teams to execute them.
2+ years of experience building ML-powered or LLM-powered products.
Hands-on experience with LLM APIs (e.g. OpenAI, Anthropic, Azure OpenAI), embeddings, vector databases, and tool use.
Hands-on experience with prompt design, context window management, and model evaluation.
Preferred
No preferred qualifications listed.