Talent.com
Mercor
DevOps Engineer AI Model EvaluatorMercor • Toronto, Ontario, Canada
No longer accepting applications
DevOps Engineer AI Model Evaluator

DevOps Engineer AI Model Evaluator

Mercor • Toronto, Ontario, Canada
17 days ago
Salary
CA$85.00 hourly
Job type
  • Full-time
Job description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco our investors include Benchmark General Catalyst Peter Thiel Adam DAngelo Larry Summers and Jack Dorsey.

Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience)
Type: Contract
Compensation: $85/hour
Location: Remote

Role Responsibilities

  • Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.
  • Review model-generated implementations involving cloud platforms Kubernetes CI/CD systems and infrastructure automation.
  • Identify bugs edge cases reliability issues and failure modes in model outputs.
  • Compare outputs from multiple frontier models to assess strengths and weaknesses.
  • Apply professional engineering judgment to realistic infrastructure engineering scenarios.

Qualifications

Must-Have

  • 2 years of professional DevOps SRE or Cloud Engineering experience.
  • Experience with AWS Azure GCP Kubernetes Terraform CI/CD pipelines or observability tooling.
  • Regular use of AI coding agents like Cursor Claude Code Codex Windsurf Gemini CLI or similar tools.
  • Ability to evaluate model-generated infrastructure and reliability engineering solutions.

Preferred

  • Experience supporting production-scale systems.

Compensation & Legal

  • $400 per accepted task.
  • Compensation is tied to accepted work.

Application Process (Takes 2030 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

  • For details about the interview process and platform information please check:
  • For any help or support reach out to:

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.


Employment Type : Full Time
Experience: years
Vacancy: 1

Create a job alert for this search

DevOps Engineer AI Model Evaluator • Toronto, Ontario, Canada

Similar jobs

Senior DevOps Engineer - Cloud-Native, Azure & AI/ML

Publicis Groupe CanadaToronto, ON, CA
Full-time

A leading communications group in Toronto is seeking a Senior DevOps Engineer to design and operate cloud-native platforms.You will support AI/ML workloads and ensure scalable solutions.This role r... Show more

 • Promoted

Remote AI Engineer - Build Next-Gen ML Solutions

Yeah! GlobalToronto, ON, CA
Remote
Full-time

A global technology firm is seeking highly skilled AI Engineers to join their team.This remote position requires proficiency in programming languages like Python and experience in developing AI mod... Show more

 • Promoted

LLM Engineer

MindlanceToronto, ON, CA
Full-time

Direct message the job poster from Mindlance.Hiring: LLM Engineers in Toronto! Be part of a fast-scaling team building Smarter, Next-Generation AI Agents alongside World-Leading AI Labs.Location: T... Show more

 • Promoted

Senior LLMOps Engineer — Cloud AI Inference + Equity

TEEMA Solutions GroupToronto, ON, CA
Full-time

A rapid-growth technology firm in Toronto is seeking a Staff LLMOps Engineer to lead the design and optimization of large language model infrastructure on the cloud.The ideal candidate has over 6 y... Show more

 • Promoted

AI Model Evaluator - Legal Expert Needed

ObsidianToronto, ON, CA
Part-time

Mercor invites you to a part-time role as a Legal Expert focusing on AI training.Your legal acumen will evaluate responses regarding employment law matters.This position, in collaboration with a pr... Show more

 • Promoted

Lead AI/ML Research Engineer - Hybrid, Growth & Impact

Thomson ReutersToronto, ON, CA
Full-time

A global technology company is seeking a Lead Research Engineer to spearhead innovative AI and ML capabilities.This role involves leading technical teams, developing large-scale data processing pip... Show more

 • Promoted

Remote R&D Biologist: AI Model Evaluation & Design

DataAnnotationMarkham, York Region, CA
Remote
Full-time

An innovative company is seeking a Research and Development Biologist to enhance AI models through rigorous evaluation and problem-solving.This exciting remote role offers flexibility in project se... Show more

 • Promoted

Senior Applied AI Engineer — Product & Revenue Systems

QEA TechMarkham, ON, CA
Full-time

Build and deploy AI systems that directly drive revenue by embedding AI across QEA’s product, sales, and marketing workflows.This role focuses on execution: turning QEA’s building data, imagery, an... Show more

 • Promoted

Intern Researcher - AI Agent Evaluation

Huawei CanadaMarkham, York Region, CA
Full-time

Huawei Canada has an immediate 4-12 month internship opening for an Intern Researcher.Established in 2014, the Distributed Scheduling and Data Engine Lab is Huawei Cloud's technical innovation cent... Show more

 • Promoted

DevOps Engineer - AI Model Evaluator

MercorToronto, Ontario, Canada
CA$85.00 hourly
Remote
Part-time
Quick Apply

Headquartered in San Francisco, our investors include.DevOps / SRE / Cloud Engineer (Coding Agent Experience).Review model-generated implementations involving.Identify bugs, edge cases, reliability... Show more

Senior DevOps Engineer - Machine Learning Focus

Layer 6 AIToronto, ON, CA
Full-time

Elevate machine learning capabilities as a Senior DevOps Engineer responsible for developing automated pipelines.Collaborate with scientists and engineers to streamline data processes.This role is ... Show more

 • Promoted

Lead DevOps Engineer for AI Projects

OncousticsToronto, ON, CA
Full-time

Join Oncoustics as a Lead DevOps Engineer and revolutionize AI/ML pipelines for liver healthcare solutions.This role involves collaborating with R&D teams and streamlining model deployment processe... Show more

 • Promoted

Remote AI Data Quality & Evaluation Engineer

Veeva SystemsToronto, ON, CA
Remote
Full-time

A leading cloud technology company in Toronto is seeking an experienced professional to validate and ensure the reliability of Veeva AI Agents.The ideal candidate should have expertise in data qual... Show more

 • Promoted

Forward Deployed Engineer in AI

Sagard Holdings Manager LPToronto, ON, CA
Full-time

Take on a pivotal role at Sagard as a Forward Deployed Engineer specializing in AI infrastructure.Help drive innovative AI solutions within our investment framework.As part of the AI team at Sagard... Show more

 • Promoted

AI Engineer

BioRenderToronto, ON, CA
Full-time

At BioRender, we don’t settle for average, and neither should you.We’re building tools that transform how scientists communicate complex ideas, and we need someone who’s ready to turn ambitious ide... Show more

 • Promoted

AI Engineer for Reliability Solutions

Mosaic.techToronto, ON, CA
Full-time

Join Rootly as an AI Engineer specializing in developing reliable AI-driven solutions for incident management.Use your LLM experience to shape the future of business reliability.As a Senior AI Engi... Show more

 • Promoted

Forward Deployed AI Engineer

EQ BankToronto, ON, CA
Full-time

We are looking for a Staff-level Forward Deployed AI Engineer to design, build, and deliver AI-powered applications that create measurable business impact.This is a hands‑on engineering role with s... Show more

 • Promoted

Lead Verification Engineer AI Hardware

TenstorrentToronto, ON, CA
Full-time

Drive innovation in AI hardware as Lead Verification Engineer with Tenstorrent, based in Austin, Toronto, or Belgrade.Lead a team focusing on testing advanced AI cores and networks.At Tenstorrent, ... Show more

 • Promoted

AI Model Evaluator - Portuguese Fluency Required

ProlificToronto, ON, CA
Full-time

Prolific is looking for Portuguese speakers to evaluate and train cutting-edge AI models, with earning potential of $30/hour.Enjoy flexible hours while working from home.As a Portuguese speaker at ... Show more

 • Promoted

AI Developer Tools Engineer at Thumbtack

ThumbtackToronto, ON, CA
Full-time

Elevate developer productivity at Thumbtack as a Senior Software Engineer specializing in AI tools.Contribute to our innovative engineering platform.As part of the Developer Experience team, you wi... Show more