Legal Consultant

About This Role

Mercor is seeking a Legal Consultant to act as a Legal AI Evaluator, guiding and assessing large language models (LLMs) in legal use cases for accuracy and clarity. This role involves prompt refinement, response annotation, and fact-checking using authoritative sources.

Responsibilities

• Write and refine prompts to guide model behavior in legal use cases
• Evaluate LLM-generated responses to legal queries for accuracy, reasoning, clarity, and completeness
• Conduct fact-checking using trusted public sources and authoritative references
• Annotate model responses by identifying strengths, areas of improvement, and factual or conceptual inaccuracies
• Assess tone, completeness, and appropriateness of responses for real-world legal scenarios
• Apply consistent evaluation standards by following clear taxonomies, benchmarks, and detailed evaluation guidelines

Requirements

• Minimum of 5 years of real-world professional experience in Law
• Associated expert degree (e.g., JD, LLB, LLM, or equivalent)
• Experience in one or more of the following sub-domains: Corporate, Business & Commercial Law; Employment, Labor & Human Rights; Public, Criminal & International Governance; Private Law
• Significant experience using large language models (LLMs) and understanding of their use
• Excellent writing skills and ability to clearly explain complex legal concepts
• Comfortable evaluating complex arguments and identifying subtle reasoning or detail-level issues

Qualifications

• Associated expert degree (e.g., JD, LLB, LLM, or equivalent)
• Minimum of 5 years of real-world professional experience in Law

Nice to Have

• Prior experience with RLHF, model evaluation, or data annotation work
• Experience writing or editing high-quality legal or analytical content
• Experience translating legal concepts for non-legal audiences
• Familiarity with evaluation rubrics, benchmarks, or quality scoring systems

Skills

* Required skills