Talent.com

Program evaluation Offres d'emploi - Toronto, ON

Créer une alerte emploi pour cette recherche

Program evaluation • toronto on

Dernière mise à jour : il y a 1 jour

AI Agent Evaluation Engineer - Remote

YO AI LabsToronto, ON, Canada
Télétravail
Temps plein
Quick Apply

AI training project by creating reinforcement learning environments that evaluate AI models on complex software engineering tasks using.You will design reproducible environments, deterministic veri... Voir plus

Program Administrator

University of TorontoToronto, ON, CA
Temps plein

Dept of Curriculum, Teaching & Learning.OISE is recognized as a global leader in graduate programs in teaching and learning, continuing teacher education, and education research.As one of the large... Voir plus

Program Movement Specialist

Rogers CommunicationsToronto, ON, CA
Temps plein +1

Are you ready to take your career to new heights and be a part of a dynamic team at Rogers Sports & Media? We believe in creativity, innovation, and collaboration in everything we do, and we ar... Voir plus

Technical Program Coordinator

Yorkville UniversityToronto, ON, CA
70 000,00 $CA par an
Temps plein
Quick Apply

Who we are At Yorkville University and Toronto Film School, we believe education is more than the pursuit of knowledge – it is a catalyst for transformation.Our mission, grounded in democrati... Voir plus

Program Lead – Afterschool Programs

National EsportsToronto, Ontario, Canada
Temps partiel

Part-time / Contract (with growth potential).We are building a new generation of afterschool programs designed to give students real exposure to modern industries—esports,... Voir plus

Software Developer – AI Research & Evaluation

TuringToronto, Ontario, Canada
200,00 $CA – 300,00 $CA par jour
Temps plein
Quick Apply

Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems.Turing support... Voir plus

Program Manager

ADECCOEtobicoke, Ontario, CA
120,00 $CA par jour
Temps plein +1
Quick Apply

Adecco is currently hiring a full-time.This role is ideal for experienced professionals with expertise in portfolio management, PMO governance, project delivery, resource planning, risk management,... Voir plus

Food Program Supervisor

Neighbours Community HomesToronto
Temps plein

The Food Program Supervisor is responsible for working within the vision, mission and relevant.Neighbours Community Homes (Neighbours) to oversee a comprehensive Food Program that addresses food in... Voir plus

Sr. Program Manager

BlueSky CommerceToronto, ON, USA
Temps plein

In this full-time position, the qualified candidate will demonstrate the following activities:.Program Manager will be responsible for the successful planning and implementation of client projects ... Voir plus

Senior Program Manager

Randstad CanadaNorth York, Ontario, CA
Temporaire
Quick Apply

We are seeking a Senior Program Manager to oversee the delivery of high-consequence IT and telecommunications projects within a 24x7 critical safety environment.This role is responsible for managin... Voir plus

Substation Program & Team Leader

Demand For HRToronto, Ontario, Canada
Temps plein

MV/HV & Substation Engineering Program & Team Leader to lead and scale our portfolio of medium-voltage (MV), high-voltage (HV), substation, protection & control, and grid-interconnection projects a... Voir plus

Program Manager

BMO Financial GroupToronto, Ontario Canada
99 429,00 $CA – 116 975,00 $CA par an
Temporaire

BMO is an organization driven by a shared Purpose: Boldly Grow the Good in business and life.It calls on members of its team, to create lasting, positive change for its customers, its communities, ... Voir plus

RQ11267 - Program Manager - Senior

Source CodeToronto, ON, Canada
139,03 $CA par jour
Temps plein
Quick Apply

RQ11267 - Program Manager - Senior.Year 6 months - 1 Possible extension.Yonge Street; 222 Jarvis Street.I&IT Programs in Public sector or Broader Public Sector (BPS).Superior knowledge of curre... Voir plus

Relief Overnight Program Assistant

RenascentToronto, Ontario, Canada
21,86 $CA par heure
Temporaire
Quick Apply

Renascent is an accredited leader in the addiction treatment field, with a legacy of serving people impacted by substance addiction for over 55 years.As a community non-profit organization, Renasce... Voir plus

Data Scientist - AI Evaluation

MercorToronto, Ontario, Canada
Télétravail
Temps partiel
Quick Apply

Headquartered in San Francisco, our investors include.Design precise, task-specific grading criteria for data science deliverables.Provide detailed written justifications for evaluations and scores... Voir plus

Program Manager 5976

FoilconToronto, Ontario, Canada
86,00 $CA par heure
Temporaire

Strategize, implement, and maintain program initiatives that adhere to strategic objectives.Formulate, organize and monitor inter-connected projects, including coordination of cross-project activit... Voir plus

SAP S/4HANA Project Experience Program-Paid Professional-Development Program

Sapsol Technologies IncToronto, ON, CA
Télétravail
Temps plein
Quick Apply

SAP-trained candidates to apply for its structured.SAP S/4HANA Project Experience Program (PAP).SAPSOL Product Innovation Lab (SPIL).This program is designed for candidates who have completed SAP t... Voir plus

DSM Evaluation Manager - EM&V

Resource InnovationsToronto, ON, CA
Télétravail
Temps plein
Quick Apply

DSM Evaluation Manager to join our expanding team.In this role, you will provide technical support and project management services within the Advisory Services Group business unit, focusing on ener... Voir plus

Program Manager

Active DynamicsToronto, ON, CA
80 000,00 $CA – 105 000,00 $CA par an
Temps plein
Quick Apply

Active Dynamics is a progressive OEM supplier that designs, tests, and manufactures unique solutions for multiple systems and multiple industries.This full-service, end-to-end capabilities are pair... Voir plus

AI Agent Evaluation Engineer - Remote

AI Agent Evaluation Engineer - Remote

YO AI LabsToronto, ON, Canada
Il y a plus de 30 jours
Type de contrat
  • Temps plein
  • Télétravail
  • Quick Apply
Description de poste

Senior Software Engineer

Job Type: Contractor (~15 hours/week)
Location: Remote

Job Summary

We are seeking experienced Senior Software Engineers to support an AI training project by creating reinforcement learning environments that evaluate AI models on complex software engineering tasks using Model Context Protocol (MCP) tools.

You will design reproducible environments, deterministic verification, and reference solutions for tasks such as bug fixing, feature implementation, codebase refactoring, and performance optimization. No prior AI experience is required.

Key Responsibilities

  • Create reinforcement learning environments for software engineering tasks.
  • Design tasks involving bug fixing, feature development, refactoring, and performance optimization.
  • Build deterministic verification systems and golden reference solutions.
  • Evaluate AI agents' ability to reason through complex codebases and use MCP tools effectively.
  • Develop realistic, reproducible software engineering scenarios.
  • Ensure tasks accurately measure coding ability, problem-solving, and tool usage.
  • Document solutions and provide clear technical feedback.

Required Skills

  • Strong proficiency in Python 3, Java, Rust, C++, or TypeScript.
  • Strong understanding of algorithms and data structures.
  • Experience with bug fixing and debugging complex software issues.
  • Proven experience in feature implementation and codebase refactoring.
  • Strong knowledge of performance optimization and tuning.
  • Excellent written and verbal communication.
  • Strong attention to detail.

Preferred Qualifications

  • Experience working with large-scale or distributed codebases.
  • Familiarity with AI/ML systems is a plus but not required.
  • Experience with rigorous code reviews and software engineering best practices.
  • Experience working effectively in remote or cross-functional teams.

Hiring Process

  1. Submit an application and screening questions.
  2. Complete an AI interview (~30 minutes).
  3. Complete a technical assessment, if required.
  4. Hiring Manager review.

Compensation

Compensation is output-based, with payment provided per task that meets project specifications. Minimum weekly submission requirements may apply.

Availability

Selected experts should be prepared to begin their first tasks within 24–48 hours of completing onboarding.