Corporate Counsel Lawyer

About This Role

Evaluate large language models for legal use cases, ensuring accuracy, clarity, and completeness of responses. This role involves prompt writing, rigorous fact-checking, and consistent application of evaluation standards.

Responsibilities

• Write and refine prompts to guide model behavior in legal use cases
• Evaluate LLM-generated responses to legal queries for accuracy, reasoning, clarity, and completeness
• Conduct fact-checking using trusted public sources and authoritative references
• Annotate model responses by identifying strengths, areas of improvement, and factual or conceptual inaccuracies
• Assess tone, completeness, and appropriateness of responses for real-world legal scenarios
• Apply consistent evaluation standards by following clear taxonomies, benchmarks, and detailed evaluation guidelines

Requirements

• Minimum of 5 years of real-world professional experience in Law
• Associated expert degree (e.g., JD, LLB, LLM, or equivalent)
• Experience in one or more of Corporate, Business & Commercial Law; Employment, Labor & Human Rights; Public, Criminal & International Governance; Private Law sub-domains
• Significant experience using large language models (LLMs) and understanding of their use
• Excellent writing skills and ability to clearly explain complex legal concepts
• Comfortable evaluating complex arguments and identifying subtle reasoning or detail-level issues

Qualifications

• JD, LLB, LLM, or equivalent
• Minimum of 5 years of real-world professional experience in Law

Nice to Have

• Prior experience with RLHF, model evaluation, or data annotation work
• Experience writing or editing high-quality legal or analytical content
• Experience translating legal concepts for non-legal audiences
• Familiarity with evaluation rubrics, benchmarks, or quality scoring systems