Talent.com
Mercor
DevOps Engineer AI Model EvaluatorMercor • Toronto, Ontario, Canada
DevOps Engineer AI Model Evaluator

DevOps Engineer AI Model Evaluator

Mercor • Toronto, Ontario, Canada
30+ days ago
Salary
CA$85.00 hourly
Job type
  • Full-time
Job description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco our investors include Benchmark General Catalyst Peter Thiel Adam DAngelo Larry Summers and Jack Dorsey.

Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience)
Type: Contract
Compensation: $85/hour
Location: Remote

Role Responsibilities

  • Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.
  • Review model-generated implementations involving cloud platforms Kubernetes CI/CD systems and infrastructure automation.
  • Identify bugs edge cases reliability issues and failure modes in model outputs.
  • Compare outputs from multiple frontier models to assess strengths and weaknesses.
  • Apply professional engineering judgment to realistic infrastructure engineering scenarios.

Qualifications

Must-Have

  • 2 years of professional DevOps SRE or Cloud Engineering experience.
  • Experience with AWS Azure GCP Kubernetes Terraform CI/CD pipelines or observability tooling.
  • Regular use of AI coding agents like Cursor Claude Code Codex Windsurf Gemini CLI or similar tools.
  • Ability to evaluate model-generated infrastructure and reliability engineering solutions.

Preferred

  • Experience supporting production-scale systems.

Compensation & Legal

  • $400 per accepted task.
  • Compensation is tied to accepted work.

Application Process (Takes 2030 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

  • For details about the interview process and platform information please check:
  • For any help or support reach out to:

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.


Employment Type : Full Time
Experience: years
Vacancy: 1
Create a job alert for this search

DevOps Engineer AI Model Evaluator • Toronto, Ontario, Canada

Similar jobs

Generative AI Engineer — Azure & LLM Solutions

GEI Consultantsmarkham, york region, Canada
Full-time

A leading engineering and consulting firm in Ontario seeks an AI Engineer to develop and deploy AI solutions.The ideal candidate will have extensive software engineering experience and skills in Ge... Show more

 • Promoted

Principal AI Solutions Developer

OpenTextrichmond hill, york region, Canada
Full-time

Become a Principal AI Solutions Developer at OpenText, leading the charge in creating sophisticated agentic AI systems.Use your expertise to enhance IT Operations security and observability.OpenTex... Show more

 • Promoted

Senior DevOps Engineer - Cloud-Native, Azure & AI/ML

Publicis Groupe CanadaToronto, ON, CA
Full-time

A leading communications group in Toronto is seeking a Senior DevOps Engineer to design and operate cloud-native platforms.You will support AI/ML workloads and ensure scalable solutions.This role r... Show more

 • Promoted

AI Platform Engineer: Build & Evaluate AI Systems

Armilla AItoronto, on, Canada
Full-time

A leading AI-focused startup in Toronto is seeking an experienced AI Engineer to shape the future of AI risk management.In this pivotal role, you will be responsible for building core AI tools and ... Show more

 • Promoted

Remote AI Engineer - Build Next-Gen ML Solutions

Yeah! GlobalToronto, ON, CA
Remote
Full-time

A global technology firm is seeking highly skilled AI Engineers to join their team.This remote position requires proficiency in programming languages like Python and experience in developing AI mod... Show more

 • Promoted

Azure DevOps Engineer

Swoonmarkham, york region, Canada
Full-time

This range is provided by Swoon.Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.Direct message the job poster from Swoon.Technical Recruiter @ S... Show more

 • Promoted

Senior Applied AI Engineer — Product & Revenue Systems

QEA TechMarkham, ON, CA
Full-time

Build and deploy AI systems that directly drive revenue by embedding AI across QEA’s product, sales, and marketing workflows.This role focuses on execution: turning QEA’s building data, imagery, an... Show more

 • Promoted

DevOps Engineer - AI Model Evaluator

MercorToronto, Ontario, Canada
CA$85.00 hourly
Remote
Part-time
Quick Apply

Headquartered in San Francisco, our investors include.DevOps / SRE / Cloud Engineer (Coding Agent Experience).Review model-generated implementations involving.Identify bugs, edge cases, reliability... Show more

Senior Researcher - Generative Modeling Expert

Huawei Technologies Canada Co., Ltd.markham, on, Canada
Full-time

Become a Senior Researcher at Huawei Canada, specializing in generative modeling and 3D computer vision.Contribute to transformative AI research within a leading lab environment.At Huawei Canada's ... Show more

 • Promoted

Senior DevOps Engineer - Machine Learning Focus

Layer 6 AIToronto, ON, CA
Full-time

Elevate machine learning capabilities as a Senior DevOps Engineer responsible for developing automated pipelines.Collaborate with scientists and engineers to streamline data processes.This role is ... Show more

 • Promoted

AI Systems Engineer – AI Model (Training & Inference)

AMDmarkham, on, Canada
Full-time

At AMD, our mission is to build great products that accelerate next-generation computing experiences—from AI and data centers, to PCs, gaming and embedded systems.Grounded in a culture of innovatio... Show more

 • Promoted

Demo Engineer - AI [32469]

Stealth AI Startuptoronto, on, Canada
Full-time

Get AI-powered advice on this job and more exclusive features.This range is provided by Stealth AI Startup.Your actual pay will be based on your skills and experience — talk with your recruiter to ... Show more

 • Promoted

Lead DevOps Engineer for AI Projects

OncousticsToronto, ON, CA
Full-time

Join Oncoustics as a Lead DevOps Engineer and revolutionize AI/ML pipelines for liver healthcare solutions.This role involves collaborating with R&D teams and streamlining model deployment processe... Show more

 • Promoted

Lead Engineer for AI Model Infrastructure

MediumToronto, ON, CA
Full-time

Step into the role of Lead Engineer focusing on building AI model infrastructure.Harness GPU resources and create self-service platforms to enhance AI deployment capabilities.As a Lead Engineer, yo... Show more

 • Promoted

AI Engineer

BioRenderToronto, ON, CA
Full-time

At BioRender, we don’t settle for average, and neither should you.We’re building tools that transform how scientists communicate complex ideas, and we need someone who’s ready to turn ambitious ide... Show more

 • Promoted

Senior DevOps Engineer, AI Infrastructure

JobtailorToronto, ON, CA
Full-time

Who we are: Financeit is a point-of-sale financing provider serving some of the largest home improvement and retail organizations in Canada.Our platform helps businesses close more sales by offerin... Show more

 • Promoted

Remote AI Data Quality & Evaluation Engineer

Veeva Systemstoronto, on, Canada
Remote
Full-time

A leading cloud technology company in Toronto is seeking an experienced professional to validate and ensure the reliability of Veeva AI Agents.The ideal candidate should have expertise in data qual... Show more

 • Promoted

Machine Learning Engineer Focused on AI Systems

HR Tech Jobtoronto, on, Canada
Full-time

Step up as a Machine Learning Engineer at Upwork, where you'll develop cutting-edge AI and reinforcement learning systems.Drive transformative changes with advanced technologies across our platform... Show more

 • Promoted

Lead Verification Engineer AI Hardware

Tenstorrenttoronto, on, Canada
Full-time

Drive innovation in AI hardware as Lead Verification Engineer with Tenstorrent, based in Austin, Toronto, or Belgrade.Lead a team focusing on testing advanced AI cores and networks.At Tenstorrent, ... Show more

 • Promoted

AI Engineer for Reliability Solutions

Mosaic.techToronto, ON, CA
Full-time

Join Rootly as an AI Engineer specializing in developing reliable AI-driven solutions for incident management.Use your LLM experience to shape the future of business reliability.As a Senior AI Engi... Show more