Talent.com
Mercor
Data Science Expert - AI EvaluationMercor • Toronto, Ontario, Canada
Data Science Expert - AI Evaluation

Data Science Expert - AI Evaluation

Mercor • Toronto, Ontario, Canada
2 days ago
Job type
  • Part-time
  • Remote
  • Quick Apply
Job description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Position: Data Science Expert
Type: Contract
Compensation: $120–$170/hour
Location: Remote

Role Responsibilities

  • Design precise, task-specific grading criteria for real-world data science deliverables such as analyses, models, dashboards, and experiment readouts.
  • Score AI-generated and human work samples against criteria with detailed written justifications for every score.
  • Apply consistent, evidence-based judgment to ensure scores are reproducible and defensible.
  • Incorporate structured feedback from senior reviewers and iterate quickly on your work.
  • Work independently and asynchronously to meet deadlines while improving AI model performance.

Qualifications

Must-Have

  • 5+ years of professional data science experience in industry.
  • Background in business operations, product, or growth data science at top-tier technology companies.
  • Deep fluency in experiment design and A/B testing, metric definition, SQL/Python analysis, and communicating findings to executive stakeholders.
  • Exceptionally strong written communication.
  • Detail-oriented, consistent, and comfortable having your judgment reviewed and calibrated against peers.

Preferred

  • Prior experience with AI training, evaluation, or human-data projects.

Application Process (Takes 20–30 mins to complete)

  • Submit your resume or relevant technical background to get started.
  • Qualified applicants may be asked to complete a brief technical assessment or submit additional information.

Resources & Support

  • For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
  • For any help or support, reach out to: support@mercor.com

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Create a job alert for this search

Data Science Expert - AI Evaluation • Toronto, Ontario, Canada

Similar jobs

Expert Data Scientist For Ai Innovations

AutoTrader.caToronto, Canada
Full-time

Drive innovation in the automotive marketplace as an Expert Data Scientist focusing on AI solutions.Initiate high-impact projects using AI/ML technologies to redefine industry standards.In this rol... Show more

 • Promoted

Remote Ai Data Quality & Evaluation Engineer - C$85,000 - C$225,000 A Year - Remote

Cloud Technology CompanyEast York, Canada
Remote
Full-time

AI Data Quality & Evaluation Engineer needed to validate and ensure the reliability of AI Agents using expertise in data quality, prompt engineering, and Python. Show more

 • Promoted

Principal AI Solutions Developer

OpenTextrichmond hill, york region, Canada
Full-time

Become a Principal AI Solutions Developer at OpenText, leading the charge in creating sophisticated agentic AI systems.Use your expertise to enhance IT Operations security and observability.OpenTex... Show more

 • Promoted

Staff Data Scientist, AI Evaluations Platform

RBCToronto, ON, CA
Full-time

RBC’s AI Group is building trusted AI capabilities for the enterprise, and evaluation is one of the core controls that makes that possible.As Staff Data Scientist, AI Evaluations, you will be a sen... Show more

 • Promoted

Applied AI Research Scientist — Multimodal Evaluation

MillionLogicsToronto, ON, CA
Full-time

A global IT solutions provider is searching for skilled Applied AI Research Scientists to contribute to AI system evaluations.Candidates should hold an MS or Ph.The role involves designing rigorous... Show more

 • Promoted

Data Science Team Lead - AI & Analytics

Ample Insight Inc.Toronto, ON, CA
Full-time

Join us as a Data Science Team Lead, focusing on AI and analytics solutions.Partner with clients' teams to implement data-driven strategies that enhance business performance.In this role, you will ... Show more

 • Promoted

Applied Scientist

ProducePayToronto, Ontario, Canada
Full-time

Waabi is a leader in Physical AI, developing autonomous driving technology that powers commercial autonomous trucks and robotaxis.With offices in Toronto, San Francisco, Dallas, and Pittsburgh, the... Show more

 • Promoted

Expert AI Engineer Driving AI Initiatives with Advanced Technologies

GEI Consultantsmarkham, york region, Canada
Full-time

Elevate your career as an AI Engineer responsible for pioneering AI solutions that transform business workflows.Focus on deploying scalable AI initiatives integrating modern technologies to meet st... Show more

 • Promoted

Data Science Manager

Lyft, Inc.Toronto, ON, CA
Full-time

At Lyft, our mission is to improve people’s lives with the world’s best transportation.To do this, we start with our own community by creating an open, inclusive, and diverse organization.Our trans... Show more

 • Promoted

Data Scientist or Decision Scientist

Venterra Realtyrichmond hill, york region, Canada
Full-time

Data Scientist or Decision Scientist.Hybrid / Corporate Office in Richmond Hill, Ontario.X% discretionary incentive target*.Technology is in the DNA of its founders.The Chairman and Co-Founder hold... Show more

 • Promoted

Strategic AI Deployment Expert at Pigment

PigmentToronto
Full-time

Take on the role of a Strategic AI Deployment Expert at Pigment, focused on crafting and implementing cutting-edge AI planning solutions for clients.Leverage your skills to redefine organizational ... Show more

 • Promoted

Data Scientist: AI & Tax Innovations

BDO CanadaToronto, ON, CA
Full-time

Elevate your career at BDO as a Data Scientist in the Tax Transformation Office, where you will apply AI and machine learning to revolutionize tax operations.This role involves a blend of tech expe... Show more

 • Promoted

Hybrid Data Scientist: Impactful Analytics & AI

VictraysToronto, ON, CA
Full-time

A leading consulting firm in Toronto is seeking an experienced Data Scientist to provide data-driven solutions in a hybrid work environment.The ideal candidate will have advanced skills in data ana... Show more

 • Promoted

Expert Data Scientist for AI Innovations

AutoTrader.caToronto, ON, CA
Full-time

Drive innovation in the automotive marketplace as an Expert Data Scientist focusing on AI solutions.Initiate high-impact projects using AI/ML technologies to redefine industry standards.In this rol... Show more

 • Promoted

Remote AI Data Quality & Evaluation Engineer

Veeva Systemstoronto, on, Canada
Remote
Full-time

A leading cloud technology company in Toronto is seeking an experienced professional to validate and ensure the reliability of Veeva AI Agents.The ideal candidate should have expertise in data qual... Show more

 • Promoted

Remote CS Expert & Data Partner for AI Q&A

TELUS Digital AI Data SolutionsToronto, ON, CA
Remote
Full-time

A leading technology firm is seeking a Computer Science Expert to join as a Freelance Data Partner.This remote role involves developing complex question-and-answer pairs in your area of expertise a... Show more

 • Promoted

Remote Data Analytics Scientist – AI & Dashboards

Handis Consulting Inc.Toronto, Ontario, Canada
Remote
Full-time

A consulting firm is seeking a Data Analytics Specialist for a one-year full-time contract in Edmonton, Canada.The role involves providing leadership on data initiatives, engaging with internal and... Show more

 • Promoted

Senior Researcher in HCI and AI Systems

Huawei Technologies Canada Co., Ltd.markham, on, Canada
Full-time

Take on a pivotal role at Huawei Canada as a Senior Researcher in Human-Computer Interaction.Focus on innovative research around agentic AI and multimodal interfaces for enhanced user experiences.T... Show more

 • Promoted

Remote Climate Data Scientist - AI-Driven Research

Crossing HurdlesToronto, ON, CA
Remote
Full-time

A referral partner specializing in climate data is seeking a Climate Data Specialist to develop and refine climate analysis tasks.This role involves creating evaluation rubrics for AI performance a... Show more

 • Promoted

Senior Researcher – AI Data Systems

Huawei CanadaMarkham, ON, CA
Full-time

Huawei Canada has an immediate 12‑month opening for a Researcher.Established in 2014, the Distributed Scheduling and Data Engine Lab is Huawei Cloud's technical innovation center in Canada.The lab ... Show more