Software EngineerMercor • Montreal (administrative region), QC, CA

Software Engineer

Mercor • Montreal (administrative region), QC, CA

2 days ago

Job type

Full-time

Part-time

Job description

About The Job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Position

Conversational AI Evaluator

Type

Full-time or Part-time Contract Work

Compensation

$45–$80 / hour

Location

Remote

Commitment

20+ hours / week

Role Responsibilities

Evaluate LLM-generated responses to coding and software engineering queries for accuracy, reasoning, clarity, and completeness.
Conduct fact‑checking using trusted public sources and authoritative references.
Execute code and validate outputs using appropriate tools to ensure accuracy.
Annotate model responses by identifying strengths, areas of improvement, and factual or conceptual inaccuracies.
Assess code quality, readability, algorithmic soundness, and explanation quality.
Apply consistent evaluation standards by following clear taxonomies, benchmarks, and detailed evaluation guidelines.

Qualifications

Must‑Have

BS, MS, or PhD in Computer Science or a closely related field.

Significant real‑world experience in software engineering or related technical roles.

Expertise in at least one relevant programming language (e.g., Python, Java, C++, JavaScript, Go, Rust).

Ability to solve HackerRank or LeetCode Medium and Hard‑level problems independently.

Experience contributing to well‑known open‑source projects, including merged pull requests.

Significant experience using LLMs while coding and understanding their strengths and failure modes.

Strong attention to detail and comfort with evaluating complex technical reasoning, identifying subtle bugs or logical flaws.

Preferred

Prior experience with RLHF, model evaluation, or data annotation work.

Track record in competitive programming.

Experience reviewing code in production environments.

Familiarity with multiple programming paradigms or ecosystems.

Experience explaining complex technical concepts to non‑expert audiences.

Application Process (Takes 20–30 mins to complete)

Upload resume

AI interview based on your resume

Submit form

Resources & Support

For details about the interview process and platform information, please check : https : / / talent.docs.mercor.com / welcome / welcome

For any help or support, reach out to : support@mercor.com

PS : Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

#J-18808-Ljbffr

Create a job alert for this search

Software Engineer • Montreal (administrative region), QC, CA

Similar jobs

Développeur(euse) de logiciel | Software Developer

Delmar International Inc. • Lachine, QC, Canada

Full-time

Relevant du Gestionnaire du développement logiciel, le / la Développeur(euse) Logiciel joue un rôle clé dans la création des plateformes et des outils qui soutiennen...Show more

Last updated: 30+ days ago • Promoted

Optical Engineer (On site - Montreal, Canada)

Astronics • Dorval, QC, Canada

Full-time

Astronics - Luminescent Systems Canada Inc.Reporting to the Director of Engineering, the incumbent will be responsible for optical design, inspection test design, automation software tools.He will ...Show more

Last updated: 30+ days ago • Promoted

Software Architect - Fully Remote

Sowelo Consulting sp. z o.o. • Montreal, QC, CA

Remote

Permanent

Quick Apply

Do you have Architecture experience with a Microsoft tech stack?.Are you able to work autonomously and have strong communication skills? Great!. Please read on as we have a role for you.We ar...Show more

Last updated: 30+ days ago

Algebra Private Tutoring Jobs L'epiphanie

Superprof • L'epiphanie, Canada

Full-time +1

Superprof is Canada's #1 tutoring platform, and we're actively recruiting passionate tutors! Whether you're a student, a professional, or simply someone who loves teaching, join the largest communi...Show more

Last updated: 30+ days ago • Promoted

Sr Software Engineer

CENTRIC SOFTWARE INC • Montréal, QC, CA

Full-time

Quick Apply

Job Title : Senior Software Engineer Location : ...Show more

Last updated: 30+ days ago

Solution Engineer

Botpress Technologies Inc. • Montreal, QC, Canada

Full-time

Help bring AI agents to companies worldwide.Over the next decade, autonomous agents will redefine how we work.Botpress allows companies to build and deploy advanced AI agents that move beyond conve...Show more

Last updated: 7 days ago • Promoted

Ingénieur logiciel senior - Plateforme central / Senior Software Engineer - Core Platform

Tait • Montreal, QC, Canada

Full-time

Créer des moments qui touchent les gens.Vous conceverez et mettrez en œuvre des fonctionnalités complexes, façonnerez des normes de codage et guiderez les décisions tech...Show more

Last updated: 30+ days ago • Promoted

Développeur logiciel - Ing. de plateforme / Software Developer - Platform Eng.

Genetec • Montreal, QC, Canada

Full-time

Vous souhaitez améliorer directement la manière dont des centaines de développeurs créent, déploient et exploitent des logiciels dans des environnements cloud, su...Show more

Last updated: 30+ days ago • Promoted

Remote C# Software Engineer - AI Trainer

SuperAnnotate • Saint-Lin-Laurentides, Quebec, CA

Remote

Full-time

This is an hourly-paid, fully remote contractor role where you will review AI-generated responses and / or generate C# / . NET engineering content, evaluating reasoning quality and step-by-step problem-...Show more

Last updated: 8 days ago

Staff Software Engineer

Valsoft Corporation • Montreal, QC, CA

Full-time

Quick Apply

Sadie is transforming how restaurants engage with their customers using AI-powered voice technology.Our intelligent voice agent, Sadie, answers phone calls 24 / 7 to take bookings, manage reservation...Show more

Last updated: 4 days ago

Enseignante, enseignant de français au secondaire - École secondaire de l'Achigan

Centre de services scolaire des Samares • Saint-Roch-de-l'Achigan, QC, Canada

Full-time

Remplacement jusqu'au 23 juin 2026.Les élèves, ça prêche par l'exemple.Viens leur servir de modèle et aide-nous à créer des petits humains qui. Le Centre de services scolaire des Samares est à la re...Show more

Last updated: 18 days ago • Promoted

Senior Software Engineer - Robotics

Vention • Montreal, QC, Canada

Full-time

Headquartered in Montreal and Berlin, Vention helps manufacturers automate their operations in record time with the only hardware and software AI-powered platform built for the factory floor.Our te...Show more

Last updated: 7 days ago • Promoted

Staff / Principal Engineer

SourceIn • Montreal, QC, Canada

Full-time

Staff / Principal Software Engineer (Fully Remote – Canada, EST Hours).Are you a seasoned engineer ready to design and lead technical excellence across teams? We're hiring a.Staff / Princ...Show more

Last updated: 23 days ago • Promoted

Software Developer – API

Amilia • Greater Montreal Area, QC, CA

Full-time

Amilia is transforming the recreation and activities market with its innovative, purpose-built SaaS platform.Designed for municipalities, community centers, nonprofits, businesses, and recreation o...Show more

Last updated: 30+ days ago

Aircraft Maintenance Engineer M2

GAL AeroStaff • Pointe-Claire, QC, Canada

Full-time

Canadian aerospace company specializing in delivering high-quality technical and operational services to the aviation industry. Our expertise includes aircraft interior, structural, manufacturing, a...Show more

Last updated: 30+ days ago • Promoted

Senior Embedded Systems Engineer

Citylogix Inc • Montréal, Quebec, Canada, H2W 1B6

Full-time

Citylogix builds mobile mapping and infrastructure intelligence systems for cities and public agencies.Our platforms combine vehicle-mounted sensor systems, embedded software, and AI-driven analyti...Show more

Last updated: 9 days ago