About the job
We’re on the lookout for a detail-oriented Senior Language Engineer to help build the next wave of capabilities of our personal AI, Copilot. Language is an increasingly important modality for how we interact with computers. As a Language Engineer on the Copilot team, you will be responsible for how Copilot wields language. You will collaboratively work with data scientists, AI researchers, engineers, designers, and product managers to discover issues, define policy, evaluate language, collect data and ultimately improve the Large Language Models that power Copilot.
Responsibilities
Craft and refine the context and prompts used to steer, train and evaluate the language models that power Microsoft Copilot for emotional and intellectual use cases.
Create evaluations and establish evaluation frameworks to measure both technical/practical performance and non-deterministic performance like EQ
Research & implement novel prompting techniques
Spin the data flywheel to extract insights from extensive language datasets to uncover issues and opportunities to improve model response quality.
Accountable to own the status of key projects, proactively identifying risks and proposing solutions to ensure timely delivery.
Qualifications
Minimum
Bachelor’s Degree in Computer Science, Philosophy, Linguistics, Psychology, Literature, or related discipline AND 1+ years experience in context engineering with foundational software engineering knowledge.
3+ years experience working in a fast-paced environment, managing multiple priorities, and adapting to changing requirements and deadlines.
Hands-on experience with prompt design, context window management, and model evaluation.
Preferred
2+ years of experience shipping consumer-facing products. You’ve brought products to market, collected user feedback, defined success metrics and iteratively improved products.
1+ years of experience in building LLM applications with familiarity in agent and orchestration frameworks, tool use, LLM evaluations and driving efficiency improvements.
Technical depth in software development, data science and machine learning. While you are not expected to write code on the critical path, you’re able to navigate and contribute to code repositories to implement context engineering improvements and write LLM scorers and classifiers to evaluate model response quality.