Freelance Agent Evaluation Engineer

About the Role
The company connects specialists with project-based AI opportunities focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment. You will be involved in building a dataset to evaluate AI coding agents, assessing how well a model tackles real-world developer tasks.
Responsibilities
You will create challenging tasks and evaluation criteria within realistic simulated environments:

• Build realistic developer environments—a virtual company with a codebase, infrastructure, and context (including tickets, documentation, and conversations) that forms a believable development history.
• Design tasks from intermediate states of these environments—crafting prompts, defining what