Talent.com

Program evaluation Jobs in North York, ON

Create a job alert for this search

Program evaluation • north york on

Last updated: 8 hours ago

AI Agent Evaluation Engineer - Remote

YO AI LabsToronto, ON, Canada
Remote
Full-time
Quick Apply

AI training project by creating reinforcement learning environments that evaluate AI models on complex software engineering tasks using.You will design reproducible environments, deterministic veri... Show more

Remote Software Developer – AI Research & Evaluation

TuringToronto, Ontario, Canada
CA$200.00–CA$300.00 daily
Remote
Full-time
Quick Apply

Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems.Turing support... Show more

Senior Program Manager

BGISToronto, ON, Canada
CA$93,972.00 yearly
Full-time

All projects to be delivered within scope, schedule, and budget according to BGIS as well as client procedural guidelines.Renovation type projects, installations, signage, and other real estate rel... Show more

Director, Program Manager

Royal Bank of Canada>TORONTO, Canada
Full-time

Lead a mission-critical program portfolio governing settlement, clearing, and middle office operations for RBC Capital Markets.You'll drive strategic technology transformation across collateral man... Show more

Program Manager 5999

FoilconToronto, Ontario, Canada
CA$90.00 hourly
Temporary

Ontario and #x27;s Public Safety Radio Network (PSRN) is one of the largest and most complex public safety radio networks in North America.It covers approximately 750,000 km2 of area.PSRN serves mo... Show more

Sessional Lecturer - KIN8330H: Program Evaluation and Applied Research for Professionals 1

University of TorontoToronto, ON, CA
Full-time

Faculty of Kinesiology & Physical Education.Faculty of Kinesiology & Physical Education.KIN8330H: Program Evaluation and Applied Research for Professionals.The emphasis of this course is on underst... Show more

 • New!

Program Manager

BlueSky CommerceToronto, ON, USA
Full-time

In this full-time position, the qualified candidate will demonstrate the following activities:.Program Manager will be responsible for the successful planning and implementation of client projects ... Show more

 • New!

Technical Program Manager / Technical Program Lead

Staffingine LLCToronto, ON, Canada
Full-time
Quick Apply

Job Title: Technical Program Manager / Technical Program Lead<br /> Job Location: Toronto, ON<br /> Job Type: Contract</b></p> <p style="margin-bottom:13px"&g... Show more

SAP Program Manager

High Cliff Partners Inc.Toronto, ON, CA
Full-time
Quick Apply

Our Toronto based client is seeking an experienced SAP Program Manager to join their project immediately and will run for 10 months, with possibility of an extension.The ideal candidate brings... Show more

Program Manager

BMO Financial GroupNorth York, ON
CA$99,429.00–CA$116,975.00 yearly
Temporary

BMO is an organization driven by a shared Purpose: Boldly Grow the Good in business and life.It calls on members of its team, to create lasting, positive change for its customers, its communities, ... Show more

Program Manager 2

Advanced Micro Devices, IncMARKHAM, Ontario, Canada
Full-time

At AMD, we believe technology can change lives for the better.It can heal us, entertain us, and make us more connected, productive, and understanding of the world around us.And we’re looking for ta... Show more

Sessional Lecturer - KIN8330H: Program Evaluation and Applied Research for Professionals 1

University of TorontoToronto, ON, CA
Full-time

Faculty of Kinesiology & Physical Education.Faculty of Kinesiology & Physical Education.KIN8330H: Program Evaluation and Applied Research for Professionals .The emphasis of this course is o... Show more

Intern Researcher - AI Agent Evaluation

Huawei Technologies Canada Co., Ltd.Markham, Ontario, CA
Full-time

Huawei Canada has an immediate 4-12 month internship opening for an Intern Researcher.Established in 2014, the Distributed Scheduling and Data Engine Lab is Huawei Cloud's technical innovation cent... Show more

Finance Program Manager

Randstad CanadaToronto, Ontario, CA
Full-time +1
Quick Apply

Our recruitment agency is currently seeking a high-performing Finance Program Manager on behalf of our client, a premier global advisory firm.In this role, you will join a growing, international Tr... Show more

Program Advisor

Toronto Film SchoolToronto, CA
CA$55,000.00 yearly
Full-time
Quick Apply

Who we are At Yorkville University and Toronto Film School, we believe education is more than the pursuit of knowledge – it is a catalyst for transformation.Our mission, grounded in democrati... Show more

Program Support Assistant

Cuso InternationalToronto, Ontario
Permanent

This Volunteer Placement is Located in:.Open to Canadian Citizens and Permanent Residents of Canada only.Are you looking to use your organizational skills to advance human rights and gender equalit... Show more

Office Document Evaluation Specialist

MercorToronto, Ontario, Canada
CA$70.00–CA$100.00 hourly
Remote
Part-time
Quick Apply

Headquartered in San Francisco, our investors include.Perform pairwise comparisons of candidate responses to select the stronger one.Assess content and rendering, including layout, formatting, and ... Show more

DSM Evaluation Manager - EM&V

Resource InnovationsToronto, ON, CA
Remote
Full-time
Quick Apply

DSM Evaluation Manager to join our expanding team.In this role, you will provide technical support and project management services within the Advisory Services Group business unit, focusing on ener... Show more

Program Manager

Active DynamicsToronto, ON, CA
CA$100,000.00 yearly
Full-time
Quick Apply

Active Dynamics is a progressive OEM supplier that designs, tests, and manufactures unique solutions for multiple systems and multiple industries.This full-service, end-to-end capabilities are pair... Show more

People also ask
AI Agent Evaluation Engineer - Remote

AI Agent Evaluation Engineer - Remote

YO AI LabsToronto, ON, Canada
14 days ago
Job type
  • Full-time
  • Remote
  • Quick Apply
Job description

Senior Software Engineer

Job Type: Contractor (~15 hours/week)
Location: Remote

Job Summary

We are seeking experienced Senior Software Engineers to support an AI training project by creating reinforcement learning environments that evaluate AI models on complex software engineering tasks using Model Context Protocol (MCP) tools.

You will design reproducible environments, deterministic verification, and reference solutions for tasks such as bug fixing, feature implementation, codebase refactoring, and performance optimization. No prior AI experience is required.

Key Responsibilities

  • Create reinforcement learning environments for software engineering tasks.
  • Design tasks involving bug fixing, feature development, refactoring, and performance optimization.
  • Build deterministic verification systems and golden reference solutions.
  • Evaluate AI agents' ability to reason through complex codebases and use MCP tools effectively.
  • Develop realistic, reproducible software engineering scenarios.
  • Ensure tasks accurately measure coding ability, problem-solving, and tool usage.
  • Document solutions and provide clear technical feedback.

Required Skills

  • Strong proficiency in Python 3, Java, Rust, C++, or TypeScript.
  • Strong understanding of algorithms and data structures.
  • Experience with bug fixing and debugging complex software issues.
  • Proven experience in feature implementation and codebase refactoring.
  • Strong knowledge of performance optimization and tuning.
  • Excellent written and verbal communication.
  • Strong attention to detail.

Preferred Qualifications

  • Experience working with large-scale or distributed codebases.
  • Familiarity with AI/ML systems is a plus but not required.
  • Experience with rigorous code reviews and software engineering best practices.
  • Experience working effectively in remote or cross-functional teams.

Hiring Process

  1. Submit an application and screening questions.
  2. Complete an AI interview (~30 minutes).
  3. Complete a technical assessment, if required.
  4. Hiring Manager review.

Compensation

Compensation is output-based, with payment provided per task that meets project specifications. Minimum weekly submission requirements may apply.

Availability

Selected experts should be prepared to begin their first tasks within 24–48 hours of completing onboarding.