Talent.com
Mercor
Analytical Evaluator - AI FeedbackMercor • Toronto, Ontario, Canada
Analytical Evaluator - AI Feedback

Analytical Evaluator - AI Feedback

Mercor • Toronto, Ontario, Canada
18 days ago
Job type
  • Part-time
  • Remote
  • Quick Apply
Job description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Position: Generalist Expert
Type: Contract
Compensation: $100/hour
Location: Remote

Role Responsibilities

  • Evaluate AI-generated responses to enhance reasoning and rigor in model outputs.
  • Provide structured written feedback to improve training data quality and downstream performance.
  • Develop clear, precise, and well-evidenced written rationales beyond surface-level observations.
  • Collaborate with AI research teams to ensure consistency and coverage across datasets.
  • Work independently and asynchronously to meet deadlines while improving AI model performance.

Qualifications

Must-Have

  • Bachelor's degree from a top-100 globally ranked university preferred.
  • Strong analytical and written communication skills.
  • Ability to work independently and follow detailed task guidelines.
  • Strong critical reading skills with the ability to identify nuance, implicit meaning, and gaps in reasoning.
  • Ability to write clear, precise, and well-evidenced written rationales.
  • Consistent and honest judgment, including the ability to give critical assessments.
  • Strict attention to detail and accurate application of structured evaluation guidelines.
  • Ability to work entirely without AI writing tools.
  • Native English fluency required.

Preferred

  • Candidates based in the United States, United Kingdom, Canada, Australia, or New Zealand.

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

  • For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
  • For any help or support, reach out to: support@mercor.com

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Create a job alert for this search

Analytical Evaluator - AI Feedback • Toronto, Ontario, Canada

Similar jobs

Innovative Analytics Engineer Driving AI and Data Quality Standards

Luxury Presencetoronto, on, Canada
Full-time

Take charge as a Staff Analytics Engineer, building AI systems that enhance how teams utilize data.Collaborate with teams to define metrics, ensuring data quality and creating impactful insights fo... Show more

 • Promoted

Remote Chemistry Specialist — AI Content QA & Evaluation

Crossing Hurdlestoronto, on, Canada
Remote
Full-time

A recruitment firm is seeking a Chemistry Expert to review and verify AI-generated content and develop chemistry problems.This remote role requires an advanced degree and strong analytical skills.T... Show more

 • Promoted

Research Associate – AI Enablement Lead

QuadReal Property Grouptoronto, on, Canada
Full-time

The Associate will report to Head of Americas Research and will be part of the Global Research team.This role sits at the intersection of artificial intelligence, data strategy, and commercial real... Show more

 • Promoted

Remote Bilingual AI Trainer & Evaluator (EN/ZH)

DataannotationToronto, ON, CA
Remote
Full-time

A tech company specializing in AI models is searching for a Bilingual Community Manager.This role requires fluency in both English and Chinese as you will train chatbots by evaluating their outputs... Show more

 • Promoted

Analytical Evaluator - AI Feedback

ObsidianToronto, ON, CA
Full-time

In this role, you will evaluate AI-generated responses and provide structured written feedback.This is a great opportunity for sharp, analytical thinkers to contribute to high-impact AI research pr... Show more

 • Promoted

AI Acceleration Lead

Credit Genietoronto, on, Canada
Full-time

We’re hiring an AI Acceleration Lead to own how AI becomes part of how every team at Credit Genie works.You’ll sit at the intersection of program management, learning & development, and applied AI—... Show more

 • Promoted

Autodesk Applied AI Analyst Position

AutodeskToronto, ON, CA
Full-time

Become a key player at Autodesk as an Applied AI Analyst.This position focuses on using artificial intelligence to innovate solutions in design and engineering.We are searching for a motivated Appl... Show more

 • Promoted

AI Product Analyst: Live Prototypes & Customer Discovery

Valsoft Corptoronto, on, Canada
Full-time

A software company in Canada is seeking an AI Business Analyst to bridge customer needs and software solutions.The role involves conducting customer discovery, building prototypes using AI, and han... Show more

 • Promoted

Senior AI Reliability & Evaluation Scientist

UpworkToronto, ON, CA
Full-time

A leading technology firm is seeking a Sr.Lead AI Research Scientist to lead research initiatives that enhance the reliability of AI systems.This role combines deep technical expertise with cross-f... Show more

 • Promoted

Senior Applied AI Engineer — Product & Revenue Systems

QEA TechMarkham, ON, CA
Full-time

Build and deploy AI systems that directly drive revenue by embedding AI across QEA’s product, sales, and marketing workflows.This role focuses on execution: turning QEA’s building data, imagery, an... Show more

 • Promoted

Senior Engineer of AI Collaboration Solutions

Fitch Ratingstoronto, on, Canada
Full-time

Elevate collaboration at Fitch Group as a Senior Engineer focused on AI innovations in Toronto.Spearhead projects with Microsoft 365 to drive impactful business outcomes.Become part of Fitch’s Mode... Show more

 • Promoted

Evaluation Lead

Waabitoronto, on, Canada
Full-time

Waabi, founded by AI visionary Raquel Urtasun, is the leader in Physical AI.With a world‑class team, we’re unlocking the next era of autonomous transportation with technology that’s powering commer... Show more

 • Promoted

Remote AI Data Quality & Evaluation Engineer

Veeva SystemsToronto, ON, CA
Remote
Full-time

A leading cloud technology company in Toronto is seeking an experienced professional to validate and ensure the reliability of Veeva AI Agents.The ideal candidate should have expertise in data qual... Show more

 • Promoted

Senior AI Quality & Evaluation Architect

IDCToronto, ON, CA
Full-time

A global technology intelligence provider in Toronto seeks a Software Engineer focusing on AI evaluation.Your role includes designing evaluation infrastructure, building automated tests, and ensuri... Show more

 • Promoted

Analyst, Advanced Analytics

Fidelity Canadatoronto, on, Canada
Full-time

Current work authorization for Canada is required for all openings.This is a regular, full-time role.You will be working on a flexible hybrid schedule as part of Fidelity’s dynamic working arrangem... Show more

 • Promoted

Cohere Senior Research Engineer in AI Evaluation

Coheretoronto, on, Canada
Full-time

Cohere seeks a Senior Research Engineer in AI Evaluation to transform the measurement of LLM capabilities.Lead the development of innovative evaluation methods and scalable tools.This role demands ... Show more

 • Promoted

Become a Luxury Brand Evaluator (Toronto, ON) - Apply Now

CXGtoronto, on, Canada
Full-time

Become a Luxury Brand Evaluator (Toronto, ON).Be among the first 25 applicants.Turn your passion for luxury into a career opportunity.Explore the world of premium brands and make a lasting impact i... Show more

 • Promoted

AI Evaluation Team Manager in Autonomous Technology

WaabiToronto, ON, CA
Full-time

Elevate autonomy performance assessments as an Evaluation Lead.Spearhead analysis efforts within a newly formed centralized team focusing on cutting-edge AI for autonomous vehicles.This leadership ... Show more

 • Promoted

AI Specialist - Applied ML Research

Xanadutoronto, on, Canada
Full-time

Xanadu’s mission is to build quantum computers that are useful and available to people everywhere.At Xanadu, we are learners, innovators, researchers, collaborators and problem solvers.We are creat... Show more

 • Promoted

Python-Driven Automotive AI Evaluator

MindriftToronto, ON, CA
Full-time

A technology consultancy offering project-based opportunities seeks automotive engineers for various tasks.Applicants should have a degree in automotive engineering, along with 3+ years of experien... Show more