LLM Business Analyst

About This Role

This role is for a detail-oriented Business Analyst to support evaluation and annotation workflows, focusing on quality, accuracy, and structured reasoning. The analyst will perform side-by-side evaluations, fact-check information, and provide clear, well-supported justifications.

Responsibilities

• Perform side-by-side (SxS) evaluations on GHD, comparing outputs and assessing quality, accuracy, and relevance
• Conduct fact-checking to validate information and identify inconsistencies or inaccuracies
• Provide detailed, well-structured justifications to support evaluation decisions
• Follow established evaluation guidelines and quality standards consistently
• Use browser-based tools and extensions to support analysis and review tasks
• Execute simple command-line instructions when required to support evaluation workflows
• Document findings clearly and communicate insights to stakeholders when needed

Requirements

• Solid comfort using a computer in a professional setting
• Experience working with browser extensions and online tools
• Ability to run basic command-line commands
• Strong analytical and critical thinking skills
• High attention to detail and ability to justify decisions clearly and objectively
• Comfortable working with evaluation frameworks, comparisons, and structured analysis
• Good written communication skills in English

Nice to Have

• Previous experience with evaluation, annotation, QA, or audit-type work
• Familiarity with data quality, content evaluation, or model/output assessments
• Background in business analysis, operations, or process improvement

Benefits