Talent.com
Mercor
DevOps Engineer - AI Model EvaluatorMercor • Toronto, Ontario, Canada
DevOps Engineer - AI Model Evaluator

DevOps Engineer - AI Model Evaluator

Mercor • Toronto, Ontario, Canada
9 hours ago
Salary
CA$85.00 hourly
Job type
  • Part-time
  • Remote
  • Quick Apply
Job description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience)
Type: Contract
Compensation: $85/hour
Location: Remote

Role Responsibilities

  • Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.
  • Review model-generated implementations involving cloud platforms, Kubernetes, CI/CD systems, observability, and infrastructure automation.
  • Identify bugs, edge cases, reliability issues, and failure modes in model outputs.
  • Compare outputs from multiple frontier models to assess their strengths and weaknesses.
  • Apply professional engineering judgment to realistic infrastructure engineering scenarios.

Qualifications

Must-Have

  • 2+ years of professional DevOps, SRE, or Cloud Engineering experience.
  • Experience with AWS, Azure, GCP, Kubernetes, Terraform, CI/CD pipelines, or observability tooling.
  • Regular use of AI coding agents such as Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools.
  • Ability to evaluate model-generated infrastructure and reliability engineering solutions.

Preferred

  • Experience supporting production-scale systems.

Compensation & Legal

  • $400 per accepted task
  • Compensation tied to accepted work.

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

  • For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
  • For any help or support, reach out to: support@mercor.com

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Create a job alert for this search

DevOps Engineer - AI Model Evaluator • Toronto, Ontario, Canada

Similar jobs

Generative AI Engineer — Azure & LLM Solutions

GEI Consultantsmarkham, york region, Canada
Full-time

A leading engineering and consulting firm in Ontario seeks an AI Engineer to develop and deploy AI solutions.The ideal candidate will have extensive software engineering experience and skills in Ge... Show more

 • Promoted

Principal AI Solutions Developer

OpenTextrichmond hill, york region, Canada
Full-time

Become a Principal AI Solutions Developer at OpenText, leading the charge in creating sophisticated agentic AI systems.Use your expertise to enhance IT Operations security and observability.OpenTex... Show more

 • Promoted

Senior DevOps Engineer - Cloud-Native, Azure & AI/ML

Publicis Groupe CanadaToronto, ON, CA
Full-time

A leading communications group in Toronto is seeking a Senior DevOps Engineer to design and operate cloud-native platforms.You will support AI/ML workloads and ensure scalable solutions.This role r... Show more

 • Promoted

Remote AI Engineer - Build Next-Gen ML Solutions

Yeah! GlobalToronto, ON, CA
Remote
Full-time

A global technology firm is seeking highly skilled AI Engineers to join their team.This remote position requires proficiency in programming languages like Python and experience in developing AI mod... Show more

 • Promoted

Intern Researcher - AI Agent Evaluation

Huawei CanadaMarkham, York Region, CA
Full-time

Huawei Canada has an immediate 4-12 month internship opening for an Intern Researcher.Established in 2014, the Distributed Scheduling and Data Engine Lab is Huawei Cloud's technical innovation cent... Show more

 • Promoted

Senior Applied AI Engineer — Product & Revenue Systems

QEA TechMarkham, ON, CA
Full-time

Build and deploy AI systems that directly drive revenue by embedding AI across QEA’s product, sales, and marketing workflows.This role focuses on execution: turning QEA’s building data, imagery, an... Show more

 • Promoted

Senior Researcher - Generative Modeling Expert

Huawei Technologies Canada Co., Ltd.markham, on, Canada
Full-time

Become a Senior Researcher at Huawei Canada, specializing in generative modeling and 3D computer vision.Contribute to transformative AI research within a leading lab environment.At Huawei Canada's ... Show more

 • Promoted

Demo Engineer - AI [32469]

Stealth AI Startuptoronto, on, Canada
Full-time

Get AI-powered advice on this job and more exclusive features.This range is provided by Stealth AI Startup.Your actual pay will be based on your skills and experience — talk with your recruiter to ... Show more

 • Promoted

Lead DevOps Engineer for AI Projects

OncousticsToronto, ON, CA
Full-time

Join Oncoustics as a Lead DevOps Engineer and revolutionize AI/ML pipelines for liver healthcare solutions.This role involves collaborating with R&D teams and streamlining model deployment processe... Show more

 • Promoted

Forward Deployed Engineer in AI

Sagard Holdings Manager LPToronto, ON, CA
Full-time

Take on a pivotal role at Sagard as a Forward Deployed Engineer specializing in AI infrastructure.Help drive innovative AI solutions within our investment framework.As part of the AI team at Sagard... Show more

 • Promoted

AI Engineer

BioRenderToronto, ON, CA
Full-time

At BioRender, we don’t settle for average, and neither should you.We’re building tools that transform how scientists communicate complex ideas, and we need someone who’s ready to turn ambitious ide... Show more

 • Promoted

Innovative AI Engineer – Pulsora Canada

PulsoraToronto, ON, CA
Full-time

Leverage your AI expertise at Pulsora as a Remote Engineer focused on developing impactful AI applications.This full-time role prioritizes cutting-edge technology for sustainability efforts.As part... Show more

 • Promoted

Senior DevOps Engineer, AI Infrastructure

JobtailorToronto, ON, CA
Full-time

Who we are: Financeit is a point-of-sale financing provider serving some of the largest home improvement and retail organizations in Canada.Our platform helps businesses close more sales by offerin... Show more

 • Promoted

Remote AI Data Quality & Evaluation Engineer

Veeva Systemstoronto, on, Canada
Remote
Full-time

A leading cloud technology company in Toronto is seeking an experienced professional to validate and ensure the reliability of Veeva AI Agents.The ideal candidate should have expertise in data qual... Show more

 • Promoted

Lead Verification Engineer AI Hardware

Tenstorrenttoronto, on, Canada
Full-time

Drive innovation in AI hardware as Lead Verification Engineer with Tenstorrent, based in Austin, Toronto, or Belgrade.Lead a team focusing on testing advanced AI cores and networks.At Tenstorrent, ... Show more

 • Promoted

Thomson Reuters Lead Engineer in AI Innovation

PowerToFlytoronto, on, Canada
Full-time

Thrive as a Lead Research Engineer at Thomson Reuters, where you will harness AI and Deep Learning to create transformative solutions for clients.Drive innovative projects in a dynamic hybrid work ... Show more

 • Promoted

AI Specialist - Applied ML Research

Xanadutoronto, on, Canada
Full-time

Xanadu’s mission is to build quantum computers that are useful and available to people everywhere.At Xanadu, we are learners, innovators, researchers, collaborators and problem solvers.We are creat... Show more

 • Promoted

AI Engineer for Reliability Solutions

Mosaic.techToronto, ON, CA
Full-time

Join Rootly as an AI Engineer specializing in developing reliable AI-driven solutions for incident management.Use your LLM experience to shape the future of business reliability.As a Senior AI Engi... Show more