Freelance Agent Evaluation Engineer
RemotePosted 2 months ago Expired
This job has expired
Looking for a job like Freelance Agent Evaluation Engineer? Upload your resume and we'll notify you when similar positions become available.
Upload Your ResumeAbout This Role
This freelance role involves evaluating and improving AI systems for leading tech companies by creating structured test cases, defining gold-standard behavior, and analyzing agent performance. It's a project-based opportunity focusing on testing and enhancing AI.
Responsibilities
- Create structured test cases simulating complex human workflows
- Define gold-standard behavior and scoring logic for agent actions
- Analyze agent logs, failure modes, and decision paths
- Work with code repositories and test frameworks to validate scenarios
- Iterate on prompts, instructions, and test cases to improve clarity and difficulty
- Ensure scenarios are production-ready, easy to run, and reusable
Requirements
- 3+ years of software development experience
- Strong Python focus
- Experience with Git and code repositories
- Comfortable with structured formats like JSON/YAML for scenario description
- Understanding core LLM limitations (hallucinations, bias, context limits)
- English proficiency - B2
Qualifications
- 3+ years of software development experience with a strong Python focus
Nice to Have
- Familiarity with Docker
Skills
* Required skills
About Mindrift
Mindrift connects specialists with AI projects from major tech innovators, unlocking the potential of Generative AI by tapping into real-world expertise from across the globe.