Senior Product Manager, Copilot AI

About This Role

This role involves driving Copilot AI model capabilities, focusing on tool-use to ensure high-quality, grounded, reliable, and cost-efficient responses. You will work at the intersection of product and research, partnering with various engineering and data science teams to execute strategy and drive measurable improvements.

Responsibilities

• Develop and execute on LLM platform strategy for Copilot to extend language model's capabilities
• Prototype approaches by steering language models to drive response quality across a wide range of scenarios
• Identify and prioritize platform, orchestration and language model issues that impact quality, factuality and safety
• Work with engineers and researchers to find a path to resolution for identified issues
• Define and build measurable evaluations with relevant datasets to demonstrate quality improvements
• Define, deploy and manage experiments in production that impact language model's tool use
• Drive measurable improvements in relevance for and engagement with Copilot users
• Partner with product teams to scale tool building
• Work with inference, agents and orchestration teams to resolve dependencies
• Own the status of key projects, proactively identifying risks and proposing solutions to ensure timely delivery

Requirements

• Bachelor's Degree AND 5+ years experience in product management OR equivalent experience
• 3+ years of experience leading ambiguous product areas, defining requirements, developing roadmaps, and working with multi-disciplinary teams to execute them
• 2+ years of experience building ML-powered or LLM-powered products
• Hands-on experience with LLM APIs (e.g. OpenAI, Anthropic, Azure OpenAI), embeddings, vector databases, and tool use
• Hands-on experience with prompt design, context window management, and model evaluation

Qualifications

• Bachelor's Degree
• 5+ years experience in product management or equivalent experience, including 3+ years leading ambiguous product areas, defining requirements, developing roadmaps, and working with multi-disciplinary teams, and 2+ years building ML-powered or LLM-powered products.

Skills

* Required skills