Talent.com
YO AI Labs
AI Agent Evaluation Engineer - RemoteYO AI Labs • Ottawa, ON, Canada
AI Agent Evaluation Engineer - Remote

AI Agent Evaluation Engineer - Remote

YO AI Labs • Ottawa, ON, Canada
Il y a 23 heures
Type de contrat
  • Temps plein
  • Télétravail
  • Quick Apply
Description de poste

Senior Software Engineer

Job Type: Contractor (~15 hours/week)
Location: Remote

Job Summary

We are seeking experienced Senior Software Engineers to support an AI training project by creating reinforcement learning environments that evaluate AI models on complex software engineering tasks using Model Context Protocol (MCP) tools.

You will design reproducible environments, deterministic verification, and reference solutions for tasks such as bug fixing, feature implementation, codebase refactoring, and performance optimization. No prior AI experience is required.

Key Responsibilities

  • Create reinforcement learning environments for software engineering tasks.
  • Design tasks involving bug fixing, feature development, refactoring, and performance optimization.
  • Build deterministic verification systems and golden reference solutions.
  • Evaluate AI agents' ability to reason through complex codebases and use MCP tools effectively.
  • Develop realistic, reproducible software engineering scenarios.
  • Ensure tasks accurately measure coding ability, problem-solving, and tool usage.
  • Document solutions and provide clear technical feedback.

Required Skills

  • Strong proficiency in Python 3, Java, Rust, C++, or TypeScript.
  • Strong understanding of algorithms and data structures.
  • Experience with bug fixing and debugging complex software issues.
  • Proven experience in feature implementation and codebase refactoring.
  • Strong knowledge of performance optimization and tuning.
  • Excellent written and verbal communication.
  • Strong attention to detail.

Preferred Qualifications

  • Experience working with large-scale or distributed codebases.
  • Familiarity with AI/ML systems is a plus but not required.
  • Experience with rigorous code reviews and software engineering best practices.
  • Experience working effectively in remote or cross-functional teams.

Hiring Process

  1. Submit an application and screening questions.
  2. Complete an AI interview (~30 minutes).
  3. Complete a technical assessment, if required.
  4. Hiring Manager review.

Compensation

Compensation is output-based, with payment provided per task that meets project specifications. Minimum weekly submission requirements may apply.

Availability

Selected experts should be prepared to begin their first tasks within 24–48 hours of completing onboarding.

Créer une alerte emploi pour cette recherche

AI Agent Evaluation Engineer - Remote • Ottawa, ON, Canada

Offres similaires

Remote Software Engineer – AI Trajectory Annotations

Weekday AI (YC W21)Ottawa, ON, CA
Télétravail
Temps partiel

A leading AI consultancy is seeking experienced Software Engineers to assist in creating training datasets for AI models.This part-time role, based in Canada, demands a minimum of 2 years' experien... Voir plus

 • Offre sponsorisée

Remote Senior Solutions Engineer — Enterprise AI & Risk

GatekeeperOttawa, ON, CA
Télétravail
Temps plein

A technology company is seeking an experienced Senior Solutions Engineer for a fully remote role in Canada.The selected candidate will be responsible for leading enterprise customer implementations... Voir plus

 • Offre sponsorisée

Remote CS Expert & Data Partner for AI Q&A

TELUS Digital AI Data SolutionsOttawa, ON, CA
Télétravail
Temps plein

A leading technology firm is seeking a Computer Science Expert to join as a Freelance Data Partner.This remote role involves developing complex question-and-answer pairs in your area of expertise a... Voir plus

 • Offre sponsorisée

Remote AI Data Quality & Evaluation Engineer

Veeva SystemsOttawa, ON, CA
Télétravail
Temps plein

A leading cloud technology company in Toronto is seeking an experienced professional to validate and ensure the reliability of Veeva AI Agents.The ideal candidate should have expertise in data qual... Voir plus

 • Offre sponsorisée

AI Deployment Engineer for Public Sector

CohereOttawa, ON, CA
Temps plein

Enable secure and efficient AI adoption in the public sector as an AI Deployment Engineer at Cohere.Work closely with clients to implement innovative solutions tailored to their operational needs.T... Voir plus

 • Offre sponsorisée

Remote Chemistry Expert (LLM Evaluation) - 34879

TuringOttawa, ON, CA
Télétravail
Temps plein

Remote Chemistry Expert (LLM Evaluation) - 34879.Remote Chemistry Expert (LLM Evaluation) - 34879.Be among the first 25 applicants.Get AI-powered advice on this job and more exclusive features.Turi... Voir plus

 • Offre sponsorisée

AI Engineer

BioRenderOttawa, ON, CA
Temps plein

At BioRender, we don’t settle for average, and neither should you.We’re building tools that transform how scientists communicate complex ideas, and we need someone who’s ready to turn ambitious ide... Voir plus

 • Offre sponsorisée

Innovative AI Engineer – Pulsora Canada

PulsoraOttawa, ON, CA
Temps plein

Leverage your AI expertise at Pulsora as a Remote Engineer focused on developing impactful AI applications.This full-time role prioritizes cutting-edge technology for sustainability efforts.As part... Voir plus

 • Offre sponsorisée

Remote Data Annotation Specialist (English & Text AI)

iMerit ScholarsOttawa, ON, CA
Télétravail
Temps plein

A leading tech organization is seeking a meticulous Data Annotation Specialist to join their AI and ML projects team.This remote, full-time position involves generating and validating datasets acro... Voir plus

 • Offre sponsorisée

Dynamic AI Agent Engineer Delivering Real-World Solutions

CrestaOttawa, ON, CA
Temps plein

Be at the forefront of AI technology by developing innovative agents that drive business efficiency and enhance customer satisfaction.In this role, leverage your skills to implement AI solutions th... Voir plus

 • Offre sponsorisée

UX Engineer: Firefly AI & Prompt Tech (Remote Canada)

Atimi SoftwareOttawa, ON, CA
Télétravail
Temps plein

A software solutions company is seeking a UX Engineer to work fully remote from Canada.The ideal candidate will have extensive experience with Web Technologies and Graphics Design, and specific exp... Voir plus

 • Offre sponsorisée

AI Solutions Engineer - Remote Opportunity

Yeah! GlobalOttawa, ON, CA
Télétravail
Temps plein

Elevate your career as an AI Solutions Engineer with a remote role for a California-based client.Focus on designing and launching innovative AI initiatives that drive business success.We're buildin... Voir plus

 • Offre sponsorisée

AI Engineer – Freelancer

TwineOttawa, ON, CA
Temps plein

Join a project focused on advancing an AI‑powered influencer marketing platform by optimizing and scaling its data pipeline.The primary objective is to enhance the performance, cost‑efficiency, and... Voir plus

 • Offre sponsorisée

RLHF Evaluation Specialist for AI Training

Rex.zoneOttawa, ON, CA
Temps plein

A tech company specializing in AI is seeking an AI Trainer based in Canada.The role involves improving large language models through RLHF-style evaluations, prompt evaluation, and data labeling.Can... Voir plus

 • Offre sponsorisée

Senior AI Engineer - Backend-Driven Agentic (Remote)

BayOne SolutionsOttawa, ON, CA
Télétravail
Temps plein

A technology firm is hiring a Senior Full Stack Engineer (Backend-Leaning) with a strong focus on backend services and AI applications.The ideal candidate will have experience in building agent-bas... Voir plus

 • Offre sponsorisée

Remote AI Software Engineer - Contractor, 10-40 hrs/wk

TuringOttawa, ON, CA
Télétravail
Temps plein

A leading research accelerator is seeking a software engineering contractor based in Canada, Ontario, to evaluate and refine AI-generated code.This role requires over 5 years of experience, includi... Voir plus

 • Offre sponsorisée

Remote AI Content Evaluator - Part-Time, Flexible Hours

Work VistaOttawa, ON, CA
Télétravail
Temps partiel

A global AI services provider is seeking a remote Content Evaluator to assess the quality of AI-generated content.This part-time role involves evaluating responses for fluency, coherence, and factu... Voir plus

 • Offre sponsorisée

AI Developer

Hanalytica GmbHOttawa, ON, CA
Temps plein

This range is provided by Hanalytica GmbH.Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.Our client is looking for an exceptional.AI and machin... Voir plus

 • Offre sponsorisée

Remote Chemistry Specialist — AI Content QA & Evaluation

Crossing HurdlesOttawa, ON, CA
Télétravail
Temps plein

A recruitment firm is seeking a Chemistry Expert to review and verify AI-generated content and develop chemistry problems.This remote role requires an advanced degree and strong analytical skills.T... Voir plus

 • Offre sponsorisée

Senior AI Engineer - Production ML for Fintech (Remote)

LumenaltaOttawa, ON, CA
Télétravail
Temps plein

A forward-thinking technology firm is looking for experienced AI Engineers to design and deploy production-grade AI systems for community banks and insurance institutions.This role offers the chanc... Voir plus