Chemistry AI Evaluator

About This Role

This role involves evaluating and refining large language model (LLM) responses related to chemistry, ensuring scientific accuracy and quantitative reasoning. It requires a deep understanding of chemistry domains and excellent writing skills to guide and annotate model behavior.

Responsibilities

• Write and refine prompts to guide model behavior in chemistry contexts
• Evaluate LLM-generated responses to chemistry-related queries for scientific accuracy and quantitative reasoning
• Conduct fact-checking using authoritative public sources and domain knowledge
• Annotate model responses by identifying strengths, areas of improvement, and inaccuracies
• Assess clarity, structure, and appropriateness of explanations for different audiences
• Apply consistent evaluation standards by following clear taxonomies and benchmarks

Requirements

• PhD in Chemistry or a closely related field
• Deep expertise in Organic & Biological Chemistry, Inorganic & Materials Chemistry, Physical & Theoretical Chemistry, or Analytical & Instrumental Chemistry
• Significant experience using large language models (LLMs)
• Excellent writing skills and strong attention to detail
• Experience reviewing or editing technical or academic writing

Qualifications

• PhD in Chemistry or a closely related field

Nice to Have

• Prior experience with RLHF, model evaluation, or data annotation work
• Experience teaching or explaining chemistry concepts to non-expert audiences
• Familiarity with evaluation rubrics or structured review frameworks

Skills

* Required skills