Talent.com
Mercor
Language Model Evaluator Fully RemoteMercor • Toronto, Ontario, Canada
Language Model Evaluator Fully Remote

Language Model Evaluator Fully Remote

Mercor • Toronto, Ontario, Canada
30+ days ago
Salary
CA$15.00 hourly
Job type
  • Full-time
  • Remote
Job description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco our investors include Benchmark General Catalyst Peter Thiel Adam DAngelo Larry Summers and Jack Dorsey.

Position: Generalist - English & Bengali
Type: Contract
Compensation: $15$20/hour
Location: Remote

Role Responsibilities

  • Conduct fact-checking using trusted public sources and external tools.
  • Generate high-quality human evaluation data by identifying response strengths areas for improvement and factual inaccuracies.
  • Assess reasoning quality clarity tone and completeness of responses.
  • Ensure model responses align with expected conversational behavior and system guidelines.
  • Work independently and asynchronously to meet deadlines while improving AI model performance.

Qualifications

Must-Have

  • Bachelors degree
  • Native speaker in Bengali
  • Significant experience using large language models (LLMs)
  • Excellent writing skills in English
  • Strong attention to detail
  • Background or experience in domains requiring structured analytical thinking (e.g. research policy analytics linguistics engineering)

Preferred

  • Prior experience with RLHF model evaluation or data annotation work
  • Experience writing or editing high-quality written content
  • Experience comparing multiple outputs and making fine-grained qualitative judgments

Application Process (Takes 2030 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

  • For details about the interview process and platform information please check:
  • For any help or support reach out to:

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.


Employment Type : Full Time
Experience: years
Vacancy: 1
Create a job alert for this search

Language Model Evaluator Fully Remote • Toronto, Ontario, Canada

Similar jobs

Lead Developer Crafting Innovative Cloud Solutions Remotely

SherwebToronto, ON, CA
Remote
Full-time

Excel as a Lead Developer in creating innovative cloud solutions while nurturing your team.This remote position emphasizes excellence in coding standards and collaborative growth.As a principal fig... Show more

 • Promoted

Ai Model Evaluator - Legal Expert Needed

ObsidianToronto, Canada
Part-time

Mercor invites you to a part-time role as a Legal Expert focusing on AI training.Your legal acumen will evaluate responses regarding employment law matters.This position, in collaboration with a pr... Show more

 • Promoted

Become a Luxury Brand Evaluator (Toronto, ON) - Apply Now

CXGToronto, ON, CA
Full-time

Become a Luxury Brand Evaluator (Toronto, ON).Be among the first 25 applicants.Turn your passion for luxury into a career opportunity.Explore the world of premium brands and make a lasting impact i... Show more

 • Promoted

Senior ML Engineer — Remote, Real‑Time Ads Pipelines

StackAdaptToronto, ON, CA
Remote
Full-time

A leading technology company is seeking Senior and Staff Machine Learning Engineers to join their Data Science team.You will be responsible for designing scalable data pipelines and implementing ML... Show more

 • Promoted

Remote Chemistry Specialist — AI Content QA & Evaluation

Crossing HurdlesToronto, ON, CA
Remote
Full-time

A recruitment firm is seeking a Chemistry Expert to review and verify AI-generated content and develop chemistry problems.This remote role requires an advanced degree and strong analytical skills.T... Show more

 • Promoted

AI Model Evaluator - Legal Expert Needed

Obsidiantoronto, on, Canada
Part-time

Mercor invites you to a part-time role as a Legal Expert focusing on AI training.Your legal acumen will evaluate responses regarding employment law matters.This position, in collaboration with a pr... Show more

 • Promoted

Remote AI Engineer - LLM Specialist

PulsoraToronto, ON, CA
Remote
Full-time

Discover an exciting career as a Remote AI Engineer specializing in Large Language Models.Play a key role in developing AI-driven software solutions in a fully remote setting.Your expertise in AI/M... Show more

 • Promoted

RLHF Evaluation Specialist for AI Training

Rex.zoneToronto, ON, CA
Full-time

A tech company specializing in AI is seeking an AI Trainer based in Canada.The role involves improving large language models through RLHF-style evaluations, prompt evaluation, and data labeling.Can... Show more

 • Promoted

Remote AI Content Evaluator - Part-Time, Flexible Hours

Work VistaToronto, ON, CA
Remote
Part-time

A leading AI content evaluation firm is seeking a remote Content Evaluator to assess AI-generated outputs for quality and relevance.This part-time role, which is 100% remote, offers flexibility in ... Show more

 • Promoted

Research Engineer (Large Language Models) - SEED

Electronic Arts Inc.Toronto, ON, CA
Full-time

SEED is a pioneering group within Electronic Arts, combining creativity with applied research.We explore, build and help define the future of interactive entertainment.We exist at EA to deliver and... Show more

 • Promoted

Online Data Analyst (Lithuanian Language) Remote

TELUS Digital AI Data SolutionsToronto, ON, CA
Remote
Part-time

Online Data Analyst (Lithuanian Language) Remote.Be among the first 25 applicants.Get AI-powered advice on this job and more exclusive features.Direct message the job poster from TELUS Digital AI D... Show more

 • Promoted

Remote Chemistry Expert (LLM Evaluation) - 34879

TuringToronto, ON, CA
Remote
Full-time

Remote Chemistry Expert (LLM Evaluation) - 34879.Remote Chemistry Expert (LLM Evaluation) - 34879.Be among the first 25 applicants.Get AI-powered advice on this job and more exclusive features.Turi... Show more

 • Promoted

Content Evaluator (Remote)

Work VistaToronto, ON, CA
Remote
Full-time

United States, United Kingdom, Canada, Ireland, Australia & New Zealand.You must be legally authorised to work in the country where you are based).We are hiring on behalf of one of our clients, a l... Show more

 • Promoted

Content Evaluation Specialist (Remote)

Work VistaToronto, ON, CA
Remote
Full-time

Content Evaluation Specialist (Remote).Remote within the United States, United Kingdom, Canada, Ireland, Australia & New Zealand (You must be legally authorised to work in the country where you are... Show more

 • Promoted

Solutions Engineer *Remote* (m/f/d)

PliantToronto, ON, CA
Remote
Full-time

Pliant is a European fintech specializing in B2B payment solutions.Our modular, API-first platform helps businesses streamline spending, improve cash flow, and integrate payments into their financi... Show more

 • Promoted

Remote Software Engineer — Train & Evaluate AI Chatbot Code

DataAnnotationToronto, Ontario, Canada
Remote
Full-time

An innovative company is seeking a skilled Software Engineer to help train AI chatbots.This exciting remote role allows you to work on your own schedule while contributing to the development of cut... Show more

 • Promoted

AI Specialist - Applied ML Research

Xanadutoronto, on, Canada
Full-time

Xanadu’s mission is to build quantum computers that are useful and available to people everywhere.At Xanadu, we are learners, innovators, researchers, collaborators and problem solvers.We are creat... Show more

 • Promoted

Teen Humor Content Evaluator - Remote Freelance Contractor

TuringToronto, ON, CA
Remote
Full-time

A leading research acceleration firm is seeking a candidate to evaluate North American teen humor through structured rating and comparative ranking.The ideal candidate should be aged 18 to 19 and p... Show more

 • Promoted

Remote AI & ML Developer — LLMs, RAG & APIs

OrganimiToronto, ON, CA
Remote
Full-time

A leading B2B SaaS company is seeking an experienced AI / Machine Learning Developer to work on innovative AI models and systems.The candidate will train and deploy large language models and relate... Show more

 • Promoted

AI Model Evaluator - Portuguese Fluency Required

ProlificToronto, ON, CA
Full-time

Prolific is looking for Portuguese speakers to evaluate and train cutting-edge AI models, with earning potential of $30/hour.Enjoy flexible hours while working from home.As a Portuguese speaker at ... Show more