Talent.com
Mercor
Analytical Evaluator AI Feedback SpecialistMercor • Toronto, Ontario, Canada
Analytical Evaluator AI Feedback Specialist

Analytical Evaluator AI Feedback Specialist

Mercor • Toronto, Ontario, Canada
30+ days ago
Salary
CA$60.00 hourly
Job type
  • Full-time
Job description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco our investors include Benchmark General Catalyst Peter Thiel Adam DAngelo Larry Summers and Jack Dorsey.

Position: Generalist Expert
Type: Contract
Compensation: $60$200/hour
Location: Remote

Role Responsibilities

  • Evaluate AI-generated responses to strengthen reasoning and rigor in model outputs.
  • Provide clear structured feedback to improve training data quality and downstream performance.
  • Develop evaluation rubrics and benchmarks for assessing AI-generated tasks.
  • Work independently and asynchronously to meet deadlines while improving AI model performance.

Qualifications

Must-Have

  • Bachelors degree from a top-100 globally ranked university preferred.
  • Strong analytical and written communication skills.
  • Ability to work independently and follow detailed task guidelines.
  • Strong critical reading skills with the ability to identify nuance and gaps in reasoning.
  • Ability to write clear precise and well-evidenced written rationales.
  • Consistent and honest judgment with critical assessment skills.
  • Strict attention to detail and accurate application of structured evaluation guidelines.
  • Ability to work entirely without AI writing tools.
  • Native English fluency required.

Preferred

  • Candidates based in the United States United Kingdom Canada Australia or New Zealand.

Application Process (Takes 2030 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

  • For details about the interview process and platform information please check:
  • For any help or support reach out to:

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.


Employment Type : Full Time
Experience: years
Vacancy: 1
Create a job alert for this search

Analytical Evaluator AI Feedback Specialist • Toronto, Ontario, Canada

Similar jobs

AI & Applied Research Leader

Thomson Reuterstoronto, on, Canada
Full-time

A leading technology firm in Toronto is seeking a Manager of AI and Applied Research.This role involves managing a high-performing team focused on state-of-the-art research in information retrieval... Show more

 • Promoted

AI Platform Engineer: Build & Evaluate AI Systems

Armilla AItoronto, on, Canada
Full-time

A leading AI-focused startup in Toronto is seeking an experienced AI Engineer to shape the future of AI risk management.In this pivotal role, you will be responsible for building core AI tools and ... Show more

 • Promoted

Remote Chemistry Specialist — AI Content QA & Evaluation

Crossing HurdlesToronto, ON, CA
Remote
Full-time

A recruitment firm is seeking a Chemistry Expert to review and verify AI-generated content and develop chemistry problems.This remote role requires an advanced degree and strong analytical skills.T... Show more

 • Promoted

Applied AI Research Scientist — Multimodal Evaluation

MillionLogicsToronto, ON, CA
Full-time

A global IT solutions provider is searching for skilled Applied AI Research Scientists to contribute to AI system evaluations.Candidates should hold an MS or Ph.The role involves designing rigorous... Show more

 • Promoted

Become a Luxury Brand Evaluator (Toronto, ON) - Apply Now

CXGToronto, ON, CA
Full-time

Become a Luxury Brand Evaluator (Toronto, ON).Be among the first 25 applicants.Turn your passion for luxury into a career opportunity.Explore the world of premium brands and make a lasting impact i... Show more

 • Promoted

RLHF Evaluation Specialist for AI Training

Rex.zoneToronto, ON, CA
Full-time

A tech company specializing in AI is seeking an AI Trainer based in Canada.The role involves improving large language models through RLHF-style evaluations, prompt evaluation, and data labeling.Can... Show more

 • Promoted

Applied Scientist

ProducePayToronto, ON, CA
Full-time

Waabi is a leader in Physical AI, developing autonomous driving technology that powers commercial autonomous trucks and robotaxis.With offices in Toronto, San Francisco, Dallas, and Pittsburgh, the... Show more

 • Promoted

Remote AI Content Evaluation Specialist

Work VistaToronto, ON, CA
Remote
Part-time

A global AI company is looking for a Content Evaluation Specialist to assess AI-generated content.This part-time, remote role requires conducting thorough evaluations and comparisons of AI response... Show more

 • Promoted

Applied AI Implementation Specialist Toronto

Pear VCToronto, ON, CA
Full-time

Join Paxos Health in Toronto as an Applied AI Implementation Specialist, focusing on transforming healthcare workflows.You'll work hands-on to structure AI outputs, enhancing patient access to medi... Show more

 • Promoted

Senior Applied AI Engineer — Product & Revenue Systems

QEA TechMarkham, ON, CA
Full-time

Build and deploy AI systems that directly drive revenue by embedding AI across QEA’s product, sales, and marketing workflows.This role focuses on execution: turning QEA’s building data, imagery, an... Show more

 • Promoted

Intern Researcher - AI Agent Evaluation

Huawei CanadaMarkham, York Region, CA
Full-time

Huawei Canada has an immediate 4-12 month internship opening for an Intern Researcher.Established in 2014, the Distributed Scheduling and Data Engine Lab is Huawei Cloud's technical innovation cent... Show more

 • Promoted

Apex Focus Group Consumer Feedback Collaborator

ApexFocusGroupToronto, ON, CA
Part-time

Become a Consumer Feedback Collaborator with Apex Focus Group, participating remotely in flexible research opportunities.Share your insights to help shape future products and services.We invite ind... Show more

 • Promoted

Content Evaluation Specialist (Remote)

Work VistaToronto, ON, CA
Remote
Full-time

Content Evaluation Specialist (Remote).Remote within the United States, United Kingdom, Canada, Ireland, Australia & New Zealand (You must be legally authorised to work in the country where you are... Show more

 • Promoted

Remote AI Data Quality & Evaluation Engineer

Veeva SystemsToronto, ON, CA
Remote
Full-time

A leading cloud technology company in Toronto is seeking an experienced professional to validate and ensure the reliability of Veeva AI Agents.The ideal candidate should have expertise in data qual... Show more

 • Promoted

Applied AI Solutions Technical Lead

CAA Club GroupMarkham, ON, CA
Full-time

CAA SCO Systems & Services Inc.At CAA Club Group, we are passionate about keeping our Members safe - whether they are on the road, at home, or travelling abroad.As a trusted Member-based organizati... Show more

 • Promoted

Analytical Evaluator - AI Feedback

MercorToronto, Ontario, Canada
CA$70.00 hourly
Remote
Part-time
Quick Apply

Headquartered in San Francisco, our investors include.Provide structured written feedback to improve.Identify nuances, implicit meanings, and gaps in reasoning in.Write clear, precise, and well-evi... Show more

AI Specialist - Applied ML Research

Xanadutoronto, on, Canada
Full-time

Xanadu’s mission is to build quantum computers that are useful and available to people everywhere.At Xanadu, we are learners, innovators, researchers, collaborators and problem solvers.We are creat... Show more

 • Promoted

AI and Applied Research Leadership Role

PowerToFlytoronto, on, Canada
Full-time

Join Thomson Reuters Labs in Toronto as a Manager of AI and Applied Research.Lead a team of talented scientists and engineers to deliver top-tier AI solutions.As the Manager, you will foster a high... Show more

 • Promoted

Senior Researcher in HCI and AI Systems

Huawei Technologies Canada Co., Ltd.markham, on, Canada
Full-time

Take on a pivotal role at Huawei Canada as a Senior Researcher in Human-Computer Interaction.Focus on innovative research around agentic AI and multimodal interfaces for enhanced user experiences.T... Show more

 • Promoted

Lead, Applied AI Research & Production

Refinitivtoronto, on, Canada
Full-time

A leading fintech organization based in Toronto is seeking an experienced Manager of Applied Research.The role involves managing a high-performing team focused on problem-solving using cutting-edge... Show more