Talent.com

Program evaluation Jobs in East York, ON

Create a job alert for this search

Program evaluation • east york on

Last updated: 11 hours ago

AI Agent Evaluation Engineer - Remote

YO AI LabsToronto, ON, Canada
Remote
Full-time
Quick Apply

AI training project by creating reinforcement learning environments that evaluate AI models on complex software engineering tasks using.You will design reproducible environments, deterministic veri... Show more

Program Lead – Afterschool Programs

National EsportsToronto, Ontario, Canada
Part-time

Part-time / Contract (with growth potential).We are building a new generation of afterschool programs designed to give students real exposure to modern industries—esports,... Show more

Director of Program Management

Waterford ExecutiveGTA, Canada
Full-time +1

Director Program Management, Global.Directly responsible for all the NA Program Managers and dotted line responsibility for Chinese Program Managers.Lead and manage all the day-to-day global progra... Show more

Program Coordinator

Kensington HealthToronto, Ontario, Canada
Full-time

At Kensington Health, we are proud to have built a community where people can bring their full selves to work.As a not-for-profit, we believe in the work we do every day.It's what motivates us, ins... Show more

Program Manager 5999

FoilconToronto, Ontario, Canada
CA$90.00 hourly
Temporary

Ontario and #x27;s Public Safety Radio Network (PSRN) is one of the largest and most complex public safety radio networks in North America.It covers approximately 750,000 km2 of area.PSRN serves mo... Show more

Technical Program Manager / Technical Program Lead

Staffingine LLCToronto, ON, Canada
Full-time
Quick Apply

Job Title: Technical Program Manager / Technical Program Lead<br /> Job Location: Toronto, ON<br /> Job Type: Contract</b></p> <p style="margin-bottom:13px"&g... Show more

 • New!

Program Advisor

University of TorontoToronto, ON, CA
Full-time +1

The Faculty of Arts & Science is the heart of Canada’s leading university and one of the most comprehensive and diverse academic divisions in the world.The strength of Arts & Science derive... Show more

Capital & Maintenance Program Analyst

Toronto HydroToronto, ON, CA
Full-time

Target Variable Performance Pay: 8%.The salary range shown above reflects the expected  compensation for this position.The final salary offered will be determined based on a holistic assessment of ... Show more

Substation Program & Team Leader

Demand For HRToronto, Ontario, Canada
Full-time

MV/HV & Substation Engineering Program & Team Leader to lead and scale our portfolio of medium-voltage (MV), high-voltage (HV), substation, protection & control, and grid-interconnection projects a... Show more

RQ11267 - Program Manager - Senior

Source CodeToronto, ON, Canada
CA$139.03 daily
Full-time
Quick Apply

RQ11267 - Program Manager - Senior.Year 6 months - 1 Possible extension.Yonge Street; 222 Jarvis Street.I&IT Programs in Public sector or Broader Public Sector (BPS).Superior knowledge of curre... Show more

Lead Product Program Manager

0076 eBay CanadaToronto
Full-time

At eBay, we're more than a global ecommerce leader — we’re changing the way the world shops and sells.Our platform empowers millions of buyers and sellers in more than 190 markets around the world.... Show more

Technical Program Manager

Royal Bank of Canada>TORONTO, Canada
Full-time

We're looking for a Technical Program Manager who will drive delivery execution for agentic AI engineering squads shipping production-grade solutions across RBC.You'll own sprint planning, velocity... Show more

Program Support Assistant

Cuso InternationalToronto, Ontario
Permanent

This Volunteer Placement is Located in:.Open to Canadian Citizens and Permanent Residents of Canada only.Are you looking to use your organizational skills to advance human rights and gender equalit... Show more

 • Promoted

Cybersecurity Program Manager - Remote

NTT DATA ServicesToronto, ON, CA
Remote
Full-time

NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us.If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.... Show more

Office Document Evaluation Specialist

MercorToronto, Ontario, Canada
CA$70.00–CA$100.00 hourly
Remote
Part-time
Quick Apply

Headquartered in San Francisco, our investors include.Perform pairwise comparisons of candidate responses to select the stronger one.Assess content and rendering, including layout, formatting, and ... Show more

Director, Program Manager

0000050007 Royal Bank of CanadaTORONTO, Ontario, Canada
Full-time

What is the opportunity?Lead a mission-critical program portfolio governing settlement, clearing, and middle office operations for RBC Capital Markets.You'll drive strategic technology transformati... Show more

Program Manager AI Activation

Canadian Tire Corporation, LimitedToronto, ON
CA$79,000.00 yearly
Full-time

Reporting into AVP, AI Activation, the.Program Manager, AI Activation.AI initiatives across the AI Activation portfolio.AI Activation, ensuring consistent delivery, governance, financial rigor, and... Show more

Program Manager - GFC Acceleration

0017 eBay Canada TechnologyToronto
Full-time

At eBay, we're more than a global ecommerce leader — we’re changing the way the world shops and sells.Our platform empowers millions of buyers and sellers in more than 190 markets around the world.... Show more

Senior Program Manager (ONSITE) JP200

P@thlion Staffing CareersToronto, ON, Canada
Full-time
Quick Apply

The Senior Program Manager (IT) is responsible for leading large‑scale, complex, multi-year technology programs that span business areas, delivery teams, and vendor partners.This role provides stra... Show more

People also ask
AI Agent Evaluation Engineer - Remote

AI Agent Evaluation Engineer - Remote

YO AI LabsToronto, ON, Canada
7 days ago
Job type
  • Full-time
  • Remote
  • Quick Apply
Job description

Senior Software Engineer

Job Type: Contractor (~15 hours/week)
Location: Remote

Job Summary

We are seeking experienced Senior Software Engineers to support an AI training project by creating reinforcement learning environments that evaluate AI models on complex software engineering tasks using Model Context Protocol (MCP) tools.

You will design reproducible environments, deterministic verification, and reference solutions for tasks such as bug fixing, feature implementation, codebase refactoring, and performance optimization. No prior AI experience is required.

Key Responsibilities

  • Create reinforcement learning environments for software engineering tasks.
  • Design tasks involving bug fixing, feature development, refactoring, and performance optimization.
  • Build deterministic verification systems and golden reference solutions.
  • Evaluate AI agents' ability to reason through complex codebases and use MCP tools effectively.
  • Develop realistic, reproducible software engineering scenarios.
  • Ensure tasks accurately measure coding ability, problem-solving, and tool usage.
  • Document solutions and provide clear technical feedback.

Required Skills

  • Strong proficiency in Python 3, Java, Rust, C++, or TypeScript.
  • Strong understanding of algorithms and data structures.
  • Experience with bug fixing and debugging complex software issues.
  • Proven experience in feature implementation and codebase refactoring.
  • Strong knowledge of performance optimization and tuning.
  • Excellent written and verbal communication.
  • Strong attention to detail.

Preferred Qualifications

  • Experience working with large-scale or distributed codebases.
  • Familiarity with AI/ML systems is a plus but not required.
  • Experience with rigorous code reviews and software engineering best practices.
  • Experience working effectively in remote or cross-functional teams.

Hiring Process

  1. Submit an application and screening questions.
  2. Complete an AI interview (~30 minutes).
  3. Complete a technical assessment, if required.
  4. Hiring Manager review.

Compensation

Compensation is output-based, with payment provided per task that meets project specifications. Minimum weekly submission requirements may apply.

Availability

Selected experts should be prepared to begin their first tasks within 24–48 hours of completing onboarding.