Java Software EngineerMercor • Montreal (administrative region), QC, CA

Java Software Engineer

Mercor • Montreal (administrative region), QC, CA

9 days ago

Job type

Full-time

Part-time

Job description

About The Job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Position

Conversational AI Evaluator

Type

Full-time or Part-time Contract Work

Compensation

$45–$80 / hour

Location

Remote

Commitment

20+ hours / week

Role Responsibilities

Evaluate LLM-generated responses to coding and software engineering queries for accuracy, reasoning, clarity, and completeness.
Conduct fact‑checking using trusted public sources and authoritative references.
Execute code and validate outputs using appropriate tools to ensure accuracy.
Annotate model responses by identifying strengths, areas of improvement, and factual or conceptual inaccuracies.
Assess code quality, readability, algorithmic soundness, and explanation quality.
Apply consistent evaluation standards by following clear taxonomies, benchmarks, and detailed evaluation guidelines.

Qualifications

Must‑Have

BS, MS, or PhD in Computer Science or a closely related field.

Significant real‑world experience in software engineering or related technical roles.

Expertise in at least one relevant programming language (e.g., Python, Java, C++, JavaScript, Go, Rust).

Ability to solve HackerRank or LeetCode Medium and Hard‑level problems independently.

Experience contributing to well‑known open‑source projects, including merged pull requests.

Significant experience using LLMs while coding and understanding their strengths and failure modes.

Strong attention to detail and comfort with evaluating complex technical reasoning, identifying subtle bugs or logical flaws.

Preferred

Prior experience with RLHF, model evaluation, or data annotation work.

Track record in competitive programming.

Experience reviewing code in production environments.

Familiarity with multiple programming paradigms or ecosystems.

Experience explaining complex technical concepts to non‑expert audiences.

Application Process (Takes 20–30 mins to complete)

Upload resume

AI interview based on your resume

Submit form

Resources & Support

For details about the interview process and platform information, please check : https : / / talent.docs.mercor.com / welcome / welcome

For any help or support, reach out to : support@mercor.com

PS : Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

#J-18808-Ljbffr

Create a job alert for this search

Java Software Engineer • Montreal (administrative region), QC, CA

Similar jobs

Software Engineer, Platform - Montreal, Canada

Speechify • Montreal, QC, Canada

Full-time

The mission of Speechify is to make sure that reading is never a barrier to learning.Over 50 million people use Speechify's text-to-speech products to turn whatever they're reading – ...Show more

Last updated: 14 days ago • Promoted

Développeur(euse) de logiciel | Software Developer

Delmar International Inc. • Lachine, QC, Canada

Full-time

Relevant du Gestionnaire du développement logiciel, le / la Développeur(euse) Logiciel joue un rôle clé dans la création des plateformes et des outils qui soutiennen...Show more

Last updated: 30+ days ago • Promoted

Optical Engineer (On site - Montreal, Canada)

Astronics • Dorval, QC, Canada

Full-time

Astronics - Luminescent Systems Canada Inc.Reporting to the Director of Engineering, the incumbent will be responsible for optical design, inspection test design, automation software tools.He will ...Show more

Last updated: 30+ days ago • Promoted

Software Architect

Randstad Digital Americas • saint-esprit, qc, ca

Full-time

Oakville, Ontario (Hybrid - 3 days onsite / week).We are seeking a pragmatic and visionary.In this role, you will be the bridge between complex business requirements and robust technological solution...Show more

Last updated: 12 hours ago • Promoted • New!

Ingénieur logiciel Java / Ingénieure logiciel Java

freelance.ca • Montreal, Canada

Full-time

Notre processus de recrutement est entièrement transparent.Si le rôle qu’on a aujourd’hui n'est pas le bon pour toi, trouvons-en un qui le soit! Travaillons ensemble pour s’assurer que tu rejoignes...Show more

Last updated: 11 days ago • Promoted

Java Developer

Targeted Talent • Montreal, QC, Canada

Permanent

We are looking for an experienced.Our client is a global enterprise company with a product that you've likely used.Unfortunately we are only able to consider candidates who either live in Canad...Show more

Last updated: 30+ days ago • Promoted

Senior Java Developer (AI applications) [#4898]

Alteo • Montreal, QC, Canada

Permanent

Alteo is looking for a Senior Java Developer (AI Applications) for a permanent position based in Montreal.Maintain and implement new features in a search engine. Development and maintenance of the u...Show more

Last updated: 18 days ago • Promoted

Applied AI Software Engineer - Simpro Software

Simpro Software • laval, qc, ca

Full-time

The Applied AI Software Engineer will be responsible for the rapid technical design and delivery of AI agents and frameworks built on top of SIMPRO FSMs. Working closely with the AI Platform Product...Show more

Last updated: 18 hours ago • Promoted • New!

Full Stack Engineer - Vedan Technologies

Vedan Technologies • saint-esprit, qc, ca

Full-time

Job Title : Full Stack Engineer.We are looking for a Full Stack Engineer who can design, build, and ship high-quality, production-ready features across the stack. You will work on modern front-end ap...Show more

Last updated: 18 hours ago • Promoted • New!

Platform Engineer (AWS) with Backend Experience - Montreal-based candidates only

Kento Health Inc • H3C, QC, Canada

Full-time

Location requirement : Applicants must be based in or around Montreal and available to work on-site at our Montreal office two days per week. We started Kento Health to revolutionize healthcare for p...Show more

Last updated: 30+ days ago • Promoted

Senior Java Software Engineer

Luxoft • terrebonne, qc, ca

Full-time

One of the world's largest providers of products and services to the energy industry has a need to develop and support enterprise information system in Oil & Gas domain. Product being developed is a...Show more

Last updated: 30+ days ago • Promoted

Algebra Private Tutoring Jobs Lanaudi

Superprof • Lanaudi, Canada

Full-time +1

Superprof is Canada's #1 tutoring platform, and we're actively recruiting passionate tutors! Whether you're a student, a professional, or simply someone who loves teaching, join the largest communi...Show more

Last updated: 30+ days ago • Promoted

Staff Software Engineer

Valsoft Corporation • Montreal, QC, CA

Full-time

Quick Apply

Sadie is transforming how restaurants engage with their customers using AI-powered voice technology.Our intelligent voice agent, Sadie, answers phone calls 24 / 7 to take bookings, manage reservation...Show more

Last updated: 7 days ago

Full stack Engineer (Java / React)

TechBiz Global GmbH • Montreal, QC, CA

Full-time

At TechBiz Global, we are providing recruitment service to our TOP clients from our portfolio.Right now, we are looking for a. Design and develop web applications using Angular or React, following i...Show more

Last updated: 30+ days ago

Senior Backend Engineer

Norbert Health • Montreal, QC, Canada

Full-time

Norbert is building autonomous robots that deliver healthcare.Our AI sensing platform mounts on mobile robots and does the work of a care team memberrounding on patients, capturing vitals without c...Show more

Last updated: 30+ days ago • Promoted

Data Entry Clerk Work From Home - Part Time Focus Group Panelists

ApexFocusGroup • Sainte-Marcelline-de-Kildare, Quebec, Canada

Remote

Full-time +1

Now accepting applicants for Focus Group studies.Earn up to $850 per week part-time working from home.Must register to see if you qualify. No Call Center Representative Agent experience needed.Call ...Show more

Last updated: 30+ days ago • Promoted

Software Engineer, Intermediate or Senior

Index • Montréal, Québec, CA

Full-time

Quick Apply

Index is an Artificial Intelligence company specializing in Web Marketing.Index is innovating the marketing business and how enterprises and people interact with it. AI technologies to help companie...Show more

Last updated: 30+ days ago

Java Full Stack Developer

Infinity Solutions • Montreal, QC, Canada

Full-time

Quick Apply

Key Responsibilities : Develop and maintain web applications using Java, Spring Boot, Hibernate.Build responsive UIs with Angular / React. Design and integrate RESTful APIs and microser...Show more

Last updated: 5 hours ago • New!