AI systems are only as trustworthy as the methods used to evaluate them. At Apple, where AI powers experiences for billions of people, getting evaluation right is not a support function—it is a foundational science. Our team, part of Apple Services Engineering, is building that scientific foundation: rigorous, scalable evaluation methodology for LLMs, agentic systems, and human-AI interaction. What makes this team unusual is its interdisciplinary core. You will work alongside measurement scientists (psychometrics, validity theory), ML researchers, and platform engineers—bringing together ML research, statistical rigor, and production engineering. We are looking for an ML Research Engineer who can move fluidly across this landscape: someone who loves implementing the latest techniques in AI, has the engineering instincts to make them robust and scalable, and thrives at the intersection of research and production.

Rapid Prototyping & Experimentation: Collaborate with research and applied scientists to translate evaluation research ideas into working prototypes—implementing methods from recent papers, building experimental pipelines, and iterating quickly to validate hypotheses in areas such as preference learning, LLM-as-judge calibration, and automated failure discovery.

Research-to-Production Bridge: Own the lifecycle of moving evaluation methods from research prototypes to production-ready systems. Refactor research code into robust, well-tested Python packages and partner with platform engineers to deploy them as scalable services, APIs, and SDK components.

Experiment Infrastructure: Design and maintain the infrastructure for running large-scale evaluation experiments—orchestrating LLM judge calls, managing datasets, tracking experiment results, and ensuring reproducibility across the team's research portfolio.

Technical Feedback & Collaboration: Serve as a critical technical partner to researchers, providing engineering perspective on feasibility, scalability, and system design. Identify opportunities where engineering improvements (parallelization, caching, smarter batching) can unlock new research directions or dramatically accelerate experimentation.

Scaling Evaluation Methods: Identify bottlenecks in evaluation workflows and engineer solutions to operate at Apple scale—optimizing for throughput, cost, and reliability when running evaluation methods across large model populations and diverse use cases.

Code Quality & Engineering Standards: Champion engineering best practices within the research workflow, including version control, automated testing, documentation, and CI/CD, raising the bar for code quality across the research-engineering boundary.

Cross-Functional Integration: Work across the research and platform engineering teams to ensure that evaluation methods integrate seamlessly with Apple's broader ML infrastructure, developer workflows, and internal tooling ecosystem.