Talent.com

Program evaluation Jobs in East York, ON

Create a job alert for this search

Program evaluation • east york on

Last updated: less than 1 hour ago

AI Agent Evaluation Engineer - Remote

YO AI LabsToronto, ON, Canada
Remote
Full-time
Quick Apply

AI training project by creating reinforcement learning environments that evaluate AI models on complex software engineering tasks using.You will design reproducible environments, deterministic veri... Show more

Remote Software Developer – AI Research & Evaluation

TuringToronto, Ontario, Canada
CA$200.00–CA$300.00 daily
Remote
Full-time
Quick Apply

Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems.Turing support... Show more

Technical Program Coordinator

Yorkville UniversityToronto, ON, CA
CA$70,000.00 yearly
Full-time
Quick Apply

Who we are At Yorkville University and Toronto Film School, we believe education is more than the pursuit of knowledge – it is a catalyst for transformation.Our mission, grounded in democrati... Show more

Program Lead – Afterschool Programs

National EsportsToronto, Ontario, Canada
Part-time

Part-time / Contract (with growth potential).We are building a new generation of afterschool programs designed to give students real exposure to modern industries—esports,... Show more

Program Manager

Cleo ConsultingToronto, ON, Canada
Full-time
Quick Apply

Assignment: RQ00229 - Program Manager - Senior</b></div> <div><b>Requisition: RQ00229</b></div> <div><b>Job Title: Program Manager - Senior</b>... Show more

 • New!

Program Coordinator

University of TorontoToronto, ON, CA
Full-time

Home to over 40 departments and institutes, the University of Toronto's Temerty Faculty of Medicine lies at the heart of the Toronto Academic Health Science Network and is a global leader in ground... Show more

 • New!

Program Manager

BlueSky CommerceToronto, ON, USA
Full-time

In this full-time position, the qualified candidate will demonstrate the following activities:.Program Manager will be responsible for the successful planning and implementation of client projects ... Show more

RQ11267 - Program Manager - Senior

Source CodeToronto, ON, Canada
CA$139.03 daily
Full-time
Quick Apply

RQ11267 - Program Manager - Senior.Year 6 months - 1 Possible extension.Yonge Street; 222 Jarvis Street.I&IT Programs in Public sector or Broader Public Sector (BPS).Superior knowledge of curre... Show more

Evaluation Specialist/Recent Grad

RemoteJobsOneToronto, Canada
CA$20.00–CA$60.00 hourly
Remote
Full-time
Quick Apply

This is a fully remote position.Role Title: Evaluation Specialist/Recent Grad.We are engaging Evaluation Specialists/Recent Grads to contribute to a high-impact project for a customer at the fronti... Show more

 • New!

Food Program Supervisor

Neighbours Community HomesToronto
Full-time

The Food Program Supervisor is responsible for working within the vision, mission and relevant.Neighbours Community Homes (Neighbours) to oversee a comprehensive Food Program that addresses food in... Show more

Program Manager 5976

FoilconToronto, Ontario, Canada
CA$86.00 hourly
Temporary

Strategize, implement, and maintain program initiatives that adhere to strategic objectives.Formulate, organize and monitor inter-connected projects, including coordination of cross-project activit... Show more

Design Evaluation Expert - Graphic

MercorToronto, Ontario, Canada
Remote
Part-time
Quick Apply

Headquartered in San Francisco, our investors include.Design precise, task-specific grading criteria for real-world design deliverables such as brand identity systems and guidelines, campaign visua... Show more

Program Advisor

Toronto Film SchoolToronto, CA
CA$55,000.00 yearly
Full-time
Quick Apply

Who we are At Yorkville University and Toronto Film School, we believe education is more than the pursuit of knowledge – it is a catalyst for transformation.Our mission, grounded in democrati... Show more

CRD Implementation Program Manager

VDart IncToronto, ON, Canada
Full-time
Quick Apply

Role: CRD Implementation Program Manager (Charles River Development) </b></p> <p><b>Minimum Experience : 15+ years </b></p> <p><b>Location: NJ, NY, L... Show more

Multiplex Graduate Program 2027

MultiplexToronto, ON, CA
CA$70,000.00–CA$80,000.00 yearly
Full-time

Who we are and what we offer: Multiplex is a global construction company with a simple purpose: to construct a better future.Our Graduate Program has a strong global reputation for being one of the... Show more

DSM Evaluation Manager - EM&V

Resource InnovationsToronto, ON, CA
Remote
Full-time
Quick Apply

DSM Evaluation Manager to join our expanding team.In this role, you will provide technical support and project management services within the Advisory Services Group business unit, focusing on ener... Show more

Program Manager

Active DynamicsToronto, ON, CA
CA$80,000.00–CA$105,000.00 yearly
Full-time
Quick Apply

Active Dynamics is a progressive OEM supplier that designs, tests, and manufactures unique solutions for multiple systems and multiple industries.This full-service, end-to-end capabilities are pair... Show more

Senior Program Manager

Randstad CanadaToronto, Ontario, CA
Temporary
Quick Apply

We are seeking a highly accomplished Senior Program Manager for an enterprise-level contract opportunity based in Toronto.In this role, you will take on a premier program leadership, strategic gove... Show more

 • New!

Director, Program Manager

0000050007 Royal Bank of CanadaTORONTO, Ontario, Canada
Full-time

What is the opportunity?Lead a mission-critical program portfolio governing settlement, clearing, and middle office operations for RBC Capital Markets.You'll drive strategic technology transformati... Show more

Program Manager AI Activation

Canadian Tire Corporation, LimitedToronto, ON
CA$79,000.00 yearly
Full-time

Reporting into AVP, AI Activation, the.Program Manager, AI Activation.AI initiatives across the AI Activation portfolio.AI Activation, ensuring consistent delivery, governance, financial rigor, and... Show more

People also ask
AI Agent Evaluation Engineer - Remote

AI Agent Evaluation Engineer - Remote

YO AI LabsToronto, ON, Canada
27 days ago
Job type
  • Full-time
  • Remote
  • Quick Apply
Job description

Senior Software Engineer

Job Type: Contractor (~15 hours/week)
Location: Remote

Job Summary

We are seeking experienced Senior Software Engineers to support an AI training project by creating reinforcement learning environments that evaluate AI models on complex software engineering tasks using Model Context Protocol (MCP) tools.

You will design reproducible environments, deterministic verification, and reference solutions for tasks such as bug fixing, feature implementation, codebase refactoring, and performance optimization. No prior AI experience is required.

Key Responsibilities

  • Create reinforcement learning environments for software engineering tasks.
  • Design tasks involving bug fixing, feature development, refactoring, and performance optimization.
  • Build deterministic verification systems and golden reference solutions.
  • Evaluate AI agents' ability to reason through complex codebases and use MCP tools effectively.
  • Develop realistic, reproducible software engineering scenarios.
  • Ensure tasks accurately measure coding ability, problem-solving, and tool usage.
  • Document solutions and provide clear technical feedback.

Required Skills

  • Strong proficiency in Python 3, Java, Rust, C++, or TypeScript.
  • Strong understanding of algorithms and data structures.
  • Experience with bug fixing and debugging complex software issues.
  • Proven experience in feature implementation and codebase refactoring.
  • Strong knowledge of performance optimization and tuning.
  • Excellent written and verbal communication.
  • Strong attention to detail.

Preferred Qualifications

  • Experience working with large-scale or distributed codebases.
  • Familiarity with AI/ML systems is a plus but not required.
  • Experience with rigorous code reviews and software engineering best practices.
  • Experience working effectively in remote or cross-functional teams.

Hiring Process

  1. Submit an application and screening questions.
  2. Complete an AI interview (~30 minutes).
  3. Complete a technical assessment, if required.
  4. Hiring Manager review.

Compensation

Compensation is output-based, with payment provided per task that meets project specifications. Minimum weekly submission requirements may apply.

Availability

Selected experts should be prepared to begin their first tasks within 24–48 hours of completing onboarding.