Talent.com
Mercor
Analytical Evaluator - AI FeedbackMercor • Toronto, Ontario, Canada
Analytical Evaluator - AI Feedback

Analytical Evaluator - AI Feedback

Mercor • Toronto, Ontario, Canada
30+ days ago
Salary
CA$70.00 hourly
Job type
  • Part-time
  • Remote
  • Quick Apply
Job description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Position: Generalist Expert
Type: Contract
Compensation: $70/hour
Location: Remote

Role Responsibilities

  • Evaluate AI-generated responses to ensure accuracy and depth in reasoning.
  • Provide structured written feedback to improve AI model outputs.
  • Identify nuances, implicit meanings, and gaps in reasoning in AI-generated content.
  • Write clear, precise, and well-evidenced rationales beyond surface-level observations.
  • Work independently and asynchronously to meet deadlines and improve AI model performance.

Qualifications

Must-Have

  • Bachelor's degree from a top-500 globally ranked university preferred.
  • Strong analytical and written communication skills.
  • Ability to work independently and follow detailed task guidelines.
  • Strong critical reading skills.
  • Native English fluency required.

Preferred

  • None specified.

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

  • For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
  • For any help or support, reach out to: support@mercor.com

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Create a job alert for this search

Analytical Evaluator - AI Feedback • Toronto, Ontario, Canada

Similar jobs

Principal AI Solutions Developer

OpenTextrichmond hill, york region, Canada
Full-time

Become a Principal AI Solutions Developer at OpenText, leading the charge in creating sophisticated agentic AI systems.Use your expertise to enhance IT Operations security and observability.OpenTex... Show more

 • Promoted

AI Platform Engineer: Build & Evaluate AI Systems

Armilla AItoronto, on, Canada
Full-time

A leading AI-focused startup in Toronto is seeking an experienced AI Engineer to shape the future of AI risk management.In this pivotal role, you will be responsible for building core AI tools and ... Show more

 • Promoted

Applied AI Research Scientist — Multimodal Evaluation

MillionLogicsToronto, ON, CA
Full-time

A global IT solutions provider is searching for skilled Applied AI Research Scientists to contribute to AI system evaluations.Candidates should hold an MS or Ph.The role involves designing rigorous... Show more

 • Promoted

Become a Luxury Brand Evaluator (Toronto, ON) - Apply Now

CXGToronto, ON, CA
Full-time

Become a Luxury Brand Evaluator (Toronto, ON).Be among the first 25 applicants.Turn your passion for luxury into a career opportunity.Explore the world of premium brands and make a lasting impact i... Show more

 • Promoted

AI Model Evaluator - Legal Expert Needed

Obsidiantoronto, on, Canada
Part-time

Mercor invites you to a part-time role as a Legal Expert focusing on AI training.Your legal acumen will evaluate responses regarding employment law matters.This position, in collaboration with a pr... Show more

 • Promoted

RLHF Evaluation Specialist for AI Training

Rex.zoneToronto, ON, CA
Full-time

A tech company specializing in AI is seeking an AI Trainer based in Canada.The role involves improving large language models through RLHF-style evaluations, prompt evaluation, and data labeling.Can... Show more

 • Promoted

AI Trainer and Statistical Analyst Role with Remote Flexibility

DataAnnotationToronto, ON, CA
Remote
Full-time

Elevate your career as a Statistical Analyst and AI Trainer in a remote environment.This role offers you the chance to influence next-generation AI systems through rigorous quantitative evaluation.... Show more

 • Promoted

Applied Scientist

ProducePayToronto, ON, CA
Full-time

Waabi is a leader in Physical AI, developing autonomous driving technology that powers commercial autonomous trucks and robotaxis.With offices in Toronto, San Francisco, Dallas, and Pittsburgh, the... Show more

 • Promoted

Senior, Applied AI Engineer

Headstart AItoronto, on, Canada
Full-time

This range is provided by Headstart AI.Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.At Headstart our mission is to bring companies into the a... Show more

 • Promoted

Applied AI Implementation Specialist Toronto

Pear VCToronto, ON, CA
Full-time

Join Paxos Health in Toronto as an Applied AI Implementation Specialist, focusing on transforming healthcare workflows.You'll work hands-on to structure AI outputs, enhancing patient access to medi... Show more

 • Promoted

Senior Applied AI Engineer — Product & Revenue Systems

QEA TechMarkham, ON, CA
Full-time

Build and deploy AI systems that directly drive revenue by embedding AI across QEA’s product, sales, and marketing workflows.This role focuses on execution: turning QEA’s building data, imagery, an... Show more

 • Promoted

Intern Researcher - AI Agent Evaluation

Huawei CanadaMarkham, York Region, CA
Full-time

Huawei Canada has an immediate 4-12 month internship opening for an Intern Researcher.Established in 2014, the Distributed Scheduling and Data Engine Lab is Huawei Cloud's technical innovation cent... Show more

 • Promoted

Remote AI Content Evaluation Specialist

Work VistaToronto, ON, CA
Remote
Part-time

A global AI company is looking for a Content Evaluation Specialist to assess AI-generated content.This part-time, remote role requires conducting thorough evaluations and comparisons of AI response... Show more

 • Promoted

Applied AI Scientist (Hybrid)

HomebaseToronto, ON, CA
Full-time

Leverage, fine‑tune, and deploy models—owning the lifecycle from experimentation to.Develop and track metrics that measure real customer impact, ensuring alignment with business outcomes.Partner cr... Show more

 • Promoted

Applied AI Engineer

Nexxa.AIToronto, ON, CA
Full-time

Our mission is to translate deep technical breakthroughs into operational reality, solving some of the hardest systems-level problems in industry.We’re looking for a Applied AI Engineer to work as ... Show more

 • Promoted

Remote AI Data Quality & Evaluation Engineer

Veeva SystemsToronto, ON, CA
Remote
Full-time

A leading cloud technology company in Toronto is seeking an experienced professional to validate and ensure the reliability of Veeva AI Agents.The ideal candidate should have expertise in data qual... Show more

 • Promoted

AI Specialist - Applied ML Research

Xanadutoronto, on, Canada
Full-time

Xanadu’s mission is to build quantum computers that are useful and available to people everywhere.At Xanadu, we are learners, innovators, researchers, collaborators and problem solvers.We are creat... Show more

 • Promoted

Senior Researcher in HCI and AI Systems

Huawei Technologies Canada Co., Ltd.markham, on, Canada
Full-time

Take on a pivotal role at Huawei Canada as a Senior Researcher in Human-Computer Interaction.Focus on innovative research around agentic AI and multimodal interfaces for enhanced user experiences.T... Show more