About the job
Prime Video is changing the way people watch movies and TV shows, with millions of titles available on-demand across devices worldwide. Within our Trust & Safety organization, we are building the future of automated content classification, leveraging large language models (LLMs) to predict maturity ratings and content descriptors at scale. This role sits at the intersection of Operations, Engineering, and Content Policy, and is critical to ensuring our customers receive accurate, consistent, and trustworthy information to make viewing decisions. We are looking for a Program Manager who will own the end-to-end prompt lifecycle for our content classification automation systems. You will deeply understand our Standard Operating Procedures (SOPs), translate nuanced policy requirements into precise prompts, rigorously test outputs, and deploy changes to production. You will influence without authority across Science, Engineering, Operations, and Policy teams to drive alignment and deliver results. This is a high-judgment role where you will make strategic tradeoff decisions between precision and recall to optimize classification quality while balancing customer experience and operational efficiency.
Responsibilities
• Prompt Lifecycle Ownership: Own end-to-end prompt engineering for content classification automation, including drafting, iterating, A/B testing, and deploying prompt changes to production environments.
• SOP Expertise & Translation: Develop deep expertise in maturity rating and content descriptor SOPs. Identify critical decision points, edge cases, and nuances that must be translated into prompt logic to ensure automated outputs mirror or is better than human judgment.
• Testing & Validation: Design and execute rigorous testing frameworks to validate prompt outputs against ground truth. Analyze false positives and false negatives, and make high-judgment calls on acceptable thresholds.
• Precision vs. Recall Tradeoffs: Make strategic decisions on where to optimize for precision (minimizing false positives) versus recall (minimizing missed content), balancing customer trust, regulatory requirements, and operational cost.
• Influence Without Authority: Drive alignment and decision-making across Science, Engineering, Operations, Legal, and Policy teams without direct reporting relationships. Build consensus on prompt strategies, escalation paths, and deployment timelines.
• Production Deployment & Monitoring: Coordinate production deployments of prompt updates, monitor real-time performance metrics, and rapidly iterate based on output quality signals.
• Continuous Improvement: Establish and track KPIs for automation accuracy, coverage, and efficiency. Identify opportunities to expand automation scope and reduce manual review dependency.
• Governance & Risk Management: Maintain program governance structures for prompt changes, including change management protocols, rollback procedures, and audit trails.
• Cross-Functional Collaboration: Partner with Science teams on model behavior, Engineering on deployment infrastructure, Operations on ground truth labeling, and Policy on evolving content standards.
• Knowledge Management & Documentation: Develop and maintain comprehensive documentation of prompt logic, testing results, decision rationale, and SOP-to-prompt mapping to enable knowledge transfer and organizational continuity.
Qualifications
Minimum
3+ years of compliance program management, legal, governance, audit, risk/loss prevention, or equivalent experience
Experience managing competing priorities and using metrics to drive business decisions
Experience communicating clearly and concisely with leadership, stakeholders, and cross-functional teams
Bachelor's degree or 3+ years of equivalent professional experience managing programs or in a technology, content moderation, or AI/ML-adjacent environment
Preferred
Experience designing or architecting (design patterns, reliability and scaling) of new and existing systems, or experience in machine learning, data mining, information retrieval, statistics or natural language processing
Experience in trust and safety compliance or risk
Experience in a test-driven and formal QA development environment, including development, staging, production (or equivalent) deployment cycles
Experience driving improvement programs in the operations, engineering and support fields
Experience with statistical methods (e.g., A/B Testing, Regression)