Talent.com
Outlier AI
AI Evaluation Engineer (Remote)Outlier AI • Quebec, Quebec, Canada
Les candidatures ne sont plus acceptées
AI Evaluation Engineer (Remote)

AI Evaluation Engineer (Remote)

Outlier AI • Quebec, Quebec, Canada
Il y a 2 jours
Type de contrat
  • Temps plein
  • Télétravail
Description de poste

About the Project

Outlier helps the world’s most innovative companies improve their AI agents by providing human feedback. Do you want to shape the future of autonomous agents like OpenClaw?

We collaborate with leading AI organizations to train Large Language Models (LLMs) to function as proactive, multi-step agents. Our projects focus on teaching these systems how to design, coordinate, and optimize complex, real-world architectural workflows.

Whether you are a passionate orchestration guru or experienced software developer — we want you to help us train the world's most advanced generative systems.

Ideal Qualifications

  • 2+ years of experience in backend engineering, AI automation, or complex systems integration.
  • Proven ability to build and maintain production-grade software with modular separation (e.g., distinct services for data parsing, logic processing, and reporting).
  • Strong command of at least two major languages (e.g., Python, JavaScript, Go, or Java) and experience working with SQL databases.
  • Practical experience building for live, non-mocked environments and handling multi-turn system interactions.
  • Outstanding attention to detail and the ability to provide clear, high-density technical feedback on complex system behaviors.

Nice to have

  • Expertise building multi-stage coordination tasks where data acquisition leads to reasoned output.
  • Hands-on experience integrating agents with live tools such as Supabase, Gmail, and various APIs to solve real-world problems.
  • High level of comfort implementing persistent state and session discovery using MEMORY.md to track agent progress.
  • Experience identifying subtle failures like privacy leaks, authority escalation, or indirect prompt injections.

Créer une alerte emploi pour cette recherche

AI Evaluation Engineer (Remote) • Quebec, Quebec, Canada

Offres similaires

Remote Engineer for AI-Driven Solutions

KnakQuebec, Capitale-Nationale, CA
Télétravail
Temps plein

Advance AI tools as a Remote Engineer focused on internal solutions.Leverage full-stack development and cloud infrastructure expertise to enhance team efficiencies and technology workflows.In this ... Voir plus

 • Offre sponsorisée

Web3 Research & Intelligence Associate (Remote - Canada) - lévis

C–CorpInvest Banklévis, qc, ca
Télétravail
Temps plein

C–CorpInvest Bank is a boutique investment firm dedicated to empowering small and medium-sized enterprises (SMEs) with tailored solutions to achieve growth and success.Our team of experienced profe... Voir plus

 • Offre sponsorisée

Lead Generative AI Engineer

Inizio Partnersquébec city, qc, ca
Temps plein

Role - Lead Generative AI Engineer.Location - Can be based anywhere in Canada.Travel - Should be open to frequent travel to client sites.We are looking for a Staff Generative AI Engineer to design,... Voir plus

 • Offre sponsorisée

Web3 Research & Intelligence Associate (Remote - Canada)

C–CorpInvest Banklévis, qc, ca
Télétravail
Temps plein

C–CorpInvest Bank is a boutique investment firm dedicated to empowering small and medium-sized enterprises (SMEs) with tailored solutions to achieve growth and success.Our team of experienced profe... Voir plus

 • Offre sponsorisée

Remote Bilingual AI Trainer & Evaluator (EN/ZH)

DataannotationQuebec, Capitale-Nationale, CA
Télétravail
Temps plein

A tech company specializing in AI models is searching for a Bilingual Community Manager.This role requires fluency in both English and Chinese as you will train chatbots by evaluating their outputs... Voir plus

 • Offre sponsorisée

DevOps Engineer (AI) - lévis

Affinitylévis, qc, ca
Temps plein

Client: Enterprise class EMR provider.On behalf of our client, Affinity is seeking a DevOps Engineer to build and evolve the infrastructure behind its flagship EMR platform.You’ll own Kubernetes-ba... Voir plus

 • Offre sponsorisée

Director of AI - lévis

People In AIlévis, qc, ca
Temps plein

Director, AI / ML (Applied AI & Agentic Systems).A scaled, product-led technology company operating at the intersection of data, AI, and vertical SaaS—focused on transforming how complex, real-worl... Voir plus

 • Offre sponsorisée

AI Development Engineer - Remote

NTT DATA, Inc.Quebec, Capitale-Nationale, CA
Télétravail
Temps plein

AI Development Engineer - Remote.We are seeking a highly motivated and skilled AI Engineer to join our team.This role focuses on designing, developing, and deploying AI-driven solutions that addres... Voir plus

 • Offre sponsorisée

AI Model Evaluation Analyst - lévis

Workerbeelévis, qc, ca
Permanent

No C2C or third-party submissions.Workerbee connects workers with employers through trusted introductions.By joining Workerbee you can be matched for project-based, contract, or permanent opportuni... Voir plus

 • Offre sponsorisée

Remote AI Data Quality & Evaluation Engineer

Veeva SystemsQuebec, Capitale-Nationale, CA
Télétravail
Temps plein

A leading cloud technology company in Toronto is seeking an experienced professional to validate and ensure the reliability of Veeva AI Agents.The ideal candidate should have expertise in data qual... Voir plus

 • Offre sponsorisée

Senior AI Engineer - Observability AI/ML (Remote)

Grafana LabsQuebec, Capitale-Nationale, CA
Télétravail
Temps plein

A leading open-source technology company is seeking a Senior AI Engineer to develop AI features for observability tools.You will collaborate with cross-functional teams to enhance user experience a... Voir plus

 • Offre sponsorisée

Remote AI UI/UX Design Evaluator

DataAnnotationQuebec, Capitale-Nationale, CA
Télétravail
Temps plein

A leading AI design firm in Canada is seeking a Product UI Designer to enhance AI capabilities through UI/UX assessment.This independent contract role allows designers to critique AI-generated visu... Voir plus

 • Offre sponsorisée

Remote Open Source AI Prototyping Engineer

MozillaQuebec, Capitale-Nationale, CA
Télétravail
Temps plein

A leading organization in open-source technology is seeking a contractor to build prototypes for open source AI infrastructure.The role involves developing working demonstrations that promote engag... Voir plus

 • Offre sponsorisée

AI Model Evaluation Analyst - saint-augustin-de-desmaures

Workerbeesaint-augustin-de-desmaures, qc, ca
Permanent

No C2C or third-party submissions.Workerbee connects workers with employers through trusted introductions.By joining Workerbee you can be matched for project-based, contract, or permanent opportuni... Voir plus

 • Offre sponsorisée

Generative AI Developer – Retail/CPG Domain - lévis

Comprehensive Resources Inclévis, qc, ca
Temps plein

Generative AI Developer (Retail/CPG).Seeking a GenAI Developer with expertise in LLMs, RAG, and prompt engineering.Build scalable AI solutions using Python, LangChain, HuggingFace, and vector DBs l... Voir plus

 • Offre sponsorisée

Remote AI Content Evaluation Specialist

Work VistaQuebec, Capitale-Nationale, CA
Télétravail
Temps partiel

A global AI company is looking for a Content Evaluation Specialist to assess AI-generated content.This part-time, remote role requires conducting thorough evaluations and comparisons of AI response... Voir plus

 • Offre sponsorisée

Generative AI Developer – Retail/CPG Domain

Comprehensive Resources Incquébec city, qc, ca
Temps plein

Generative AI Developer (Retail/CPG).Seeking a GenAI Developer with expertise in LLMs, RAG, and prompt engineering.Build scalable AI solutions using Python, LangChain, HuggingFace, and vector DBs l... Voir plus

 • Offre sponsorisée

AI Integration & Onboarding Specialist - lévis

SayVo AIlévis, qc, ca
Temps plein

AI Integration & Onboarding Specialist.Contractor (with full-time potential).SayVo AI is a fast-moving startup that builds AI employees that call, text, email, and close the loop in the CRM.In a nu... Voir plus

 • Offre sponsorisée

Lead Generative AI Engineer - lévis

Inizio Partnerslévis, qc, ca
Temps plein

Role - Lead Generative AI Engineer.Location - Can be based anywhere in Canada.Travel - Should be open to frequent travel to client sites.We are looking for a Staff Generative AI Engineer to design,... Voir plus

 • Offre sponsorisée

Remote AI Evaluation Scenario Designer

MindriftQuebec, Capitale-Nationale, CA
Télétravail
Temps plein

A leading AI solutions company is looking for an expert to design and evaluate test scenarios for AI agents.The ideal candidate will have a Bachelor's or Master's degree in a relevant field, experi... Voir plus