In this role, you will work cross-functionally with Researchers and Engineers on Agent Evals and Quality to ensure that we have the best quality of agents for GenAI model improvement and product developments. You will collaborate with agent platform and model teams to leverage user signals and metrics to improve model performance. focusing on the development, evaluation, and optimization of AI agentic systems—specificly LLM-based agents designed to perform complex, multi-step tasks and workflows.

Construct quantitative benchmarks and automated evaluation frameworks (including LLM-as-a-judge) to measure agent capabilities in reasoning, planning, and tool use.

Create and optimize data mixes extracted from user feedback for training, fine-tuning agents to enhance performance on specific tool-use tasks.

Analyze agent behavior to identify failure modes, edge cases, and performance bottlenecks, turning these insights into actionable improvements.