Full Stack Developer

About This Role

Contribute to well-known open-source Python repositories for AI benchmarking projects. Responsibilities include evaluating the performance of coding agents, assessing outputs against technical criteria, and documenting findings.

Responsibilities

• Work with well-known, well-documented open-source Python repositories to support AI benchmarking projects
• Evaluate the performance of coding agents across open-ended software engineering tasks
• Assess agent outputs against predefined technical and quality criteria
• Apply real-world open-source development judgment to identify strengths, weaknesses, and edge cases in AI-generated code
• Document findings clearly to support model evaluation and improvement
• Collaborate asynchronously with research and engineering teams throughout the project lifecycle

Requirements

• Strong experience contributing to or maintaining open-source Python repositories
• Deep proficiency in Python
• Familiarity with common open-source development workflows
• Ability to evaluate code quality, correctness, and maintainability
• Strong analytical skills with high attention to detail
• Clear written communication for documenting evaluations and insights
• Comfortable working independently in a remote, project-based environment

Skills

* Required skills