Talent.com
Mercor
Language Model Evaluator - Fully Remote | Upto $20/hr Part-timeMercor • Toronto, Ontario, Canada
Language Model Evaluator - Fully Remote | Upto $20/hr Part-time

Language Model Evaluator - Fully Remote | Upto $20/hr Part-time

Mercor • Toronto, Ontario, Canada
27 days ago
Salary
CA$15.00 hourly
Job type
  • Part-time
  • Remote
  • Quick Apply
Job description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Position: Generalist - English & Bengali
Type: Contract
Compensation: $15–$20/hour
Location: Remote

Role Responsibilities

  • Conduct fact-checking using trusted public sources and external tools.
  • Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuracies.
  • Assess reasoning quality, clarity, tone, and completeness of responses.
  • Ensure model responses align with expected conversational behavior and system guidelines.
  • Work independently and asynchronously to meet deadlines while improving AI model performance.

Qualifications

Must-Have

  • Bachelor's degree
  • Native speaker in Bengali
  • Significant experience using large language models (LLMs)
  • Excellent writing skills in English
  • Strong attention to detail
  • Background or experience in domains requiring structured analytical thinking (e.g., research, policy, analytics, linguistics, engineering)

Preferred

  • Prior experience with RLHF, model evaluation, or data annotation work
  • Experience writing or editing high-quality written content
  • Experience comparing multiple outputs and making fine-grained qualitative judgments

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

  • For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
  • For any help or support, reach out to: support@mercor.com

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Create a job alert for this search

Language Model Evaluator - Fully Remote | Upto $20/hr Part-time • Toronto, Ontario, Canada

Similar jobs

Experimentation Advocate

GrowthBookToronto, ON, CA
Full-time

We are looking for a data scientist to join our team to drive the conversation around digital experimentation and GrowthBook, work directly with active and prospective GrowthBook customers to ensur... Show more

 • Promoted

AI/ML Engineer Co‑op — Build Hiring AI (Remote)

Northeading TechnologiesToronto, ON, CA
Remote
Full-time

A technology company in Canada is seeking an AI/ML Engineer intern to work closely with the founding team to create impactful AI systems for their hiring platform.You will prototype AI agents, fine... Show more

 • Promoted

Senior Rust Code Reviewer for AI Training Evaluations

G2i Inc.Toronto, ON, CA
Full-time

A leading AI training company in Canada is seeking a skilled Code Reviewer with deep expertise in Rust.You will review evaluations of AI-generated Rust code to ensure quality and correctness.The id... Show more

 • Promoted

Remote AI Evaluation Scenario Designer

MindriftToronto, ON, CA
Remote
Full-time

A leading AI solutions company is looking for an expert to design and evaluate test scenarios for AI agents.The ideal candidate will have a Bachelor's or Master's degree in a relevant field, experi... Show more

 • Promoted

Remote QA Scenario Writer for AI Evaluation - Flexible

MindriftToronto, ON, CA
Remote
Part-time

A technology consulting firm in Canada is seeking an Evaluation Scenario Writer for a remote, part-time role.This position requires a strong QA background in complex testing environments and offers... Show more

 • Promoted

RLHF Evaluation Specialist for AI Training

Rex.zoneToronto, ON, CA
Full-time

A tech company specializing in AI is seeking an AI Trainer based in Canada.The role involves improving large language models through RLHF-style evaluations, prompt evaluation, and data labeling.Can... Show more

 • Promoted

Remote and Hybrid Independent Contractor Technology Lawyers (Canada) - MT>Align, a division of

MT>Align, a division of McCarthy Tétrault LLPmarkham, on, ca
Remote
Full-time

Work for yourself, but with us!.Align, a division of McCarthy Tétrault LLP, pairs high caliber independent contract lawyers with our clients who need legal support.Align, successful applicants will... Show more

 • Promoted

Italian Fluency Needed for AI Evaluators

ProlificToronto, ON, CA
Full-time

Become an AI evaluator with Prolific, focusing on your Italian fluency.Earn $30 per hour while engaging in remote tasks that improve AI systems with flexible hours.We are looking for individuals wh... Show more

 • Promoted

Research Engineer (Large Language Models) - SEED

Electronic Arts Inc.Toronto, ON, CA
Full-time

SEED is a pioneering group within Electronic Arts, combining creativity with applied research.We explore, build and help define the future of interactive entertainment.We exist at EA to deliver and... Show more

 • Promoted

AI Code Reviewer & Systems Evaluation Engineer 1

Lifted, an Upwork CompanyToronto, ON, CA
Full-time

An enterprise client is currently seeking experienced software engineers to contribute to improving advanced AI systems through human feedback.This work supports leading AI organizations in trainin... Show more

 • Promoted

Content Evaluator (Remote)

Work VistaToronto, ON, CA
Remote
Full-time

United States, United Kingdom, Canada, Ireland, Australia & New Zealand.You must be legally authorised to work in the country where you are based).We are hiring on behalf of one of our clients, a l... Show more

 • Promoted

Remote AI Data Quality & Evaluation Engineer

Veeva SystemsToronto, ON, CA
Remote
Full-time

A leading cloud technology company in Toronto is seeking an experienced professional to validate and ensure the reliability of Veeva AI Agents.The ideal candidate should have expertise in data qual... Show more

 • Promoted

Implementation Specialist Toronto, Ontario, Canada Remote

AppspaceToronto, ON, CA
Remote
Full-time

At Appspace, we’re passionate about creating better work experiences for people everywhere, and we’re looking for people that feel the same way.Our global office locations and flexible work culture... Show more

 • Promoted

AI Security Evaluator & Red Team Operator (Remote)

DataAnnotationToronto, ON, CA
Remote
Full-time

A tech company specializing in cybersecurity is seeking experienced professionals to help train AI models.In this remote role, you will evaluate AI-generated security content, solve technical cyber... Show more

 • Promoted

Autonomy Evaluation Lead (Remote) — Build Metrics & Dashboards

WaabiToronto, ON, CA
Remote
Full-time

A leading AI technology firm is seeking a hands-on leader to build a centralized Evaluation team focused on the performance analysis of autonomous systems.The successful candidate will have 6+ year... Show more

 • Promoted

Online Survey Participant: Work Remote and Earn Up To $25 Per Survey

Earn HausEast Gwillimbury, ON, CA
Remote
Full-time +1

Looking for people to participate in taking online surveys for Fortune 500 brands.All you need to do is complete online surveys by sharing your opinion.You will help influence brand decisions on se... Show more

 • Promoted

Design Researcher

Creative NicheGreater Toronto Area, Canada, Canada
Full-time

We're Hiring: Design Researcher (12-Month+ Contract | Toronto | 4 Days In-Office | 1 Day Remote).We’re on the lookout for a passionate Design Researcher to join a high-impact digital team within on... Show more

 • Promoted

Web3 Product Analyst - richmond hill

SINOFYrichmond hill, on, ca
Full-time

We are a team of passionate Web3 enthusiasts turned venture capitalists, dedicated to investing in transformative startups.Our mission is to cut through the hype and support the creation of financi... Show more

 • Promoted

Remote ML Engineer - Real-Time Pipelines & Inference

StackAdaptToronto, ON, CA
Remote
Full-time

A leading marketing technology firm is looking for a Machine Learning Engineer to join its Data Science team.This remote-first position involves designing scalable data pipelines and implementing a... Show more

 • Promoted

QA Mannual - Canada (Remote)

RELQ TECHNOLOGIESToronto, ON, CA
Remote
Full-time +1

Job Title: Senior Manual QA Engineer.Remote (Canada) – PST preferred (Vancouver, BC candidates highly preferred).Product-based company – SaaS HRMS applications for the public sector.Our client is a... Show more