Talent.com
Outlier AI
AI Evaluation Engineer (Remote)Outlier AI • Calgary, Alberta, Canada
AI Evaluation Engineer (Remote)

AI Evaluation Engineer (Remote)

Outlier AI • Calgary, Alberta, Canada
Il y a 15 heures
Type de contrat
  • Temps plein
  • Télétravail
Description de poste

About the Project

Outlier helps the world’s most innovative companies improve their AI agents by providing human feedback. Do you want to shape the future of autonomous agents like OpenClaw?

We collaborate with leading AI organizations to train Large Language Models (LLMs) to function as proactive, multi-step agents. Our projects focus on teaching these systems how to design, coordinate, and optimize complex, real-world architectural workflows.

Whether you are a passionate orchestration guru or experienced software developer — we want you to help us train the world's most advanced generative systems.

Ideal Qualifications

  • 2+ years of experience in backend engineering, AI automation, or complex systems integration.
  • Proven ability to build and maintain production-grade software with modular separation (e.g., distinct services for data parsing, logic processing, and reporting).
  • Strong command of at least two major languages (e.g., Python, JavaScript, Go, or Java) and experience working with SQL databases.
  • Practical experience building for live, non-mocked environments and handling multi-turn system interactions.
  • Outstanding attention to detail and the ability to provide clear, high-density technical feedback on complex system behaviors.

Nice to have

  • Expertise building multi-stage coordination tasks where data acquisition leads to reasoned output.
  • Hands-on experience integrating agents with live tools such as Supabase, Gmail, and various APIs to solve real-world problems.
  • High level of comfort implementing persistent state and session discovery using MEMORY.md to track agent progress.
  • Experience identifying subtle failures like privacy leaks, authority escalation, or indirect prompt injections.

Créer une alerte emploi pour cette recherche

AI Evaluation Engineer (Remote) • Calgary, Alberta, Canada

Offres similaires

Generative AI Developer – Retail/CPG Domain

Comprehensive Resources Inccalgary, ab, ca
Temps plein

Generative AI Developer (Retail/CPG).Seeking a GenAI Developer with expertise in LLMs, RAG, and prompt engineering.Build scalable AI solutions using Python, LangChain, HuggingFace, and vector DBs l... Voir plus

 • Offre sponsorisée

Remote AI Data Quality & Evaluation Engineer

Veeva SystemsCalgary, AB, CA
Télétravail
Temps plein

A leading cloud technology company in Toronto is seeking an experienced professional to validate and ensure the reliability of Veeva AI Agents.The ideal candidate should have expertise in data qual... Voir plus

 • Offre sponsorisée

Remote AI Evaluation Scenario Designer

MindriftCalgary, AB, CA
Télétravail
Temps plein

A leading AI solutions company is looking for an expert to design and evaluate test scenarios for AI agents.The ideal candidate will have a Bachelor's or Master's degree in a relevant field, experi... Voir plus

 • Offre sponsorisée

Remote AI UI/UX Design Evaluator

DataAnnotationCalgary, Alberta, Canada
Télétravail
Temps plein

A leading AI design firm in Canada is seeking a Product UI Designer to enhance AI capabilities through UI/UX assessment.This independent contract role allows designers to critique AI-generated visu... Voir plus

 • Offre sponsorisée

Remote Research Engineer - Decentralized AI Systems

Yotta LabsCalgary, AB, CA
Télétravail
Temps plein

A leading tech company is seeking a Research Engineer specializing in decentralized AI systems.The role involves designing efficient workload orchestration for AI applications across a global netwo... Voir plus

 • Offre sponsorisée

Climate AI Research Engineer — Remote/Hybrid

ErodeCalgary, AB, CA
Télétravail
Temps plein

A climate-tech startup is seeking a Machine Learning Research Engineer to develop AI models for weather forecasting and energy production.The role allows for remote or hybrid work in Calgary or Cha... Voir plus

 • Offre sponsorisée

Web3 Research & Intelligence Associate (Remote - Canada)

C–CorpInvest Bankcalgary, ab, ca
Télétravail
Temps plein

C–CorpInvest Bank is a boutique investment firm dedicated to empowering small and medium-sized enterprises (SMEs) with tailored solutions to achieve growth and success.Our team of experienced profe... Voir plus

 • Offre sponsorisée

Remote AI Content Evaluation Specialist

Work VistaCalgary, AB, CA
Télétravail
Temps partiel

A global AI company is looking for a Content Evaluation Specialist to assess AI-generated content.This part-time, remote role requires conducting thorough evaluations and comparisons of AI response... Voir plus

 • Offre sponsorisée

Remote Chemistry Expert (LLM Evaluation) - 34879

TuringCalgary, AB, CA
Télétravail
Temps plein

Remote Chemistry Expert (LLM Evaluation) - 34879.Remote Chemistry Expert (LLM Evaluation) - 34879.Be among the first 25 applicants.Get AI-powered advice on this job and more exclusive features.Turi... Voir plus

 • Offre sponsorisée

Remote Engineer for AI-Driven Solutions

KnakCalgary, AB, CA
Télétravail
Temps plein

Advance AI tools as a Remote Engineer focused on internal solutions.Leverage full-stack development and cloud infrastructure expertise to enhance team efficiencies and technology workflows.In this ... Voir plus

 • Offre sponsorisée

AI Engineer (Remote)

LumenaltaCalgary, AB, CA
Télétravail
Temps plein

At Lumenalta, we partner with forward‑thinking organizations to build technology solutions that scale, delight users, and accelerate business growth.Our global teams bring curiosity, commitment, an... Voir plus

 • Offre sponsorisée

Remote AI Development Engineer Crafting Innovative Solutions

NTT Data Americas, Inc.Calgary, AB, CA
Télétravail
Temps plein

Elevate your career as a Remote AI Development Engineer focused on cutting-edge AI solutions.Utilize your expertise in machine learning and data engineering to transform complex business challenges... Voir plus

 • Offre sponsorisée

RLHF Evaluation Specialist for AI Training

Rex.zoneCalgary, AB, CA
Temps plein

A tech company specializing in AI is seeking an AI Trainer based in Canada.The role involves improving large language models through RLHF-style evaluations, prompt evaluation, and data labeling.Can... Voir plus

 • Offre sponsorisée

AI Model Evaluation Analyst

Workerbeecalgary, ab, ca
Permanent

No C2C or third-party submissions.Workerbee connects workers with employers through trusted introductions.By joining Workerbee you can be matched for project-based, contract, or permanent opportuni... Voir plus

 • Offre sponsorisée

Remote Bilingual AI Trainer & Evaluator (EN/ZH)

DataannotationCalgary, AB, CA
Télétravail
Temps plein

A tech company specializing in AI models is searching for a Bilingual Community Manager.This role requires fluency in both English and Chinese as you will train chatbots by evaluating their outputs... Voir plus

 • Offre sponsorisée

Innovative Senior AI Quality Engineer for Cutting-Edge Evaluations

RoofrCalgary, AB, CA
Temps plein

Be at the forefront of AI quality as a Senior AI Quality Engineer.Your mission will be to implement comprehensive testing standards while enjoying the flexibility of a remote-first workplace.This r... Voir plus

 • Offre sponsorisée

Senior AI Engineer - Observability AI/ML (Remote)

Grafana LabsCalgary, AB, CA
Télétravail
Temps plein

A leading open-source technology company is seeking a Senior AI Engineer to develop AI features for observability tools.You will collaborate with cross-functional teams to enhance user experience a... Voir plus

 • Offre sponsorisée

AI Development Engineer - Remote

NTT DATA, Inc.Calgary, AB, CA
Télétravail
Temps plein

AI Development Engineer - Remote.We are seeking a highly motivated and skilled AI Engineer to join our team.This role focuses on designing, developing, and deploying AI-driven solutions that addres... Voir plus

 • Offre sponsorisée

Remote Open Source AI Prototyping Engineer

MozillaCalgary, AB, CA
Télétravail
Temps plein

A leading organization in open-source technology is seeking a contractor to build prototypes for open source AI infrastructure.The role involves developing working demonstrations that promote engag... Voir plus

 • Offre sponsorisée

AI Engineer

NTT DATA North AmericaCalgary, AB, CA
Temps plein

NTT DATA strives to hire exceptional, innovative, and passionate individuals who want to grow with us.We are an inclusive, adaptable, and forward‑thinking organization.We are seeking a highly motiv... Voir plus