AI Coding Model Evaluation Engineer
Turing Enterprises, Inc. • toronto, on, Canada
6 hours ago
Job description
Evaluate AI-generated code with Python or TypeScript expertise as an independent contractor supporting Turing’s model improvement work.
This experienced software engineering role focuses on reviewing coding agents across real-world repositories. You will assess correctness, robustness, maintainability, and agent behavior, then create evaluation signals that improve coding models. The work includes collaboration with researchers and engineers.
Key Responsibilities:
- Evaluate AI-generated code across real-world software repositories
- Review agent behavior, tool usage, and code changes
- Identify technical errors and recurring model failure modes
- Create rubrics and evaluation criteria for coding tasks
- Produce preference data and actionable findings
Requirements:
- Five or more years of software engineering experience
- Strong Python, TypeScript, JavaScript, Go, or production language proficiency
- Experience with substantial real-world codebases
- Strong code-review skills and technical judgment
- Experience using modern LLMs or AI coding tools
Apply software engineering judgment, code review, and written communication to improve Turing’s AI coding models.
#J-18808-Ljbffr