User testing Jobs in Collingwood, ON

If you are a senior risk executive who thrives on strategic judgment, regulatory nuance, enterprise-wide risk thinking, and high-stakes decision-making, this is a unique opportunity to contribute directly to how the next generation of AI systems reason about risk. We are looking for Chief Risk Officer professionals who understand enterprise risk management, financial risk, operational risk, model risk, cyber risk, governance, controls, regulatory expectations, stress testing, and board-level risk communication.

In this role, you will challenge and evaluate advanced language models on Chief Risk Officer topics such as risk appetite frameworks, capital and liquidity risk, third-party risk management, operational resilience, fraud risk, regulatory remediation, model governance, scenario analysis, internal controls, and executive risk reporting. Your work will help strengthen how AI systems interpret complex risk tradeoffs, communicate uncertainty, and produce practical, defensible recommendations for senior business leaders.

Key Responsibilities :

• Evaluate AI-generated responses for accuracy, depth, clarity, and practical usefulness across Chief Risk Officer and enterprise risk management scenarios.
• Challenge advanced language models with realistic CRO-level prompts involving risk appetite, governance, controls, regulatory expectations, financial risk, operational resilience, cyber risk, and board reporting.
• Review and refine AI-generated prompts, answers, rationales, and rubrics to ensure they reflect sound risk management judgment and executive-level reasoning.
• Identify gaps, hallucinations, weak assumptions, oversimplifications, and missing risk considerations in AI-generated technical content.
• Provide structured feedback that helps improve model behavior, factual reliability, reasoning quality, and communication standards.
• Help shape AI communication standards for risk topics, including appropriate treatment of uncertainty, escalation thresholds, regulatory sensitivity, and defensible recommendations.
• Support benchmarking efforts by developing, reviewing, and calibrating evaluation tasks that test AI performance on complex CRO and risk leadership topics.
• Assess whether AI outputs appropriately distinguish between strategic, financial, operational, compliance, cyber, reputational, and model risk considerations.
• Contribute subject matter expertise to improve how models handle risk tradeoffs, governance language, mitigation planning, and executive decision support.

Your Profile :

• 7+ years of senior-level experience in enterprise risk management, risk leadership, financial services risk, corporate risk oversight, regulatory risk, or a related executive risk function.
• Significant hands-on work with enterprise risk frameworks, risk appetite statements, board risk reporting, controls testing, risk governance, regulatory examinations, or enterprise-wide risk assessments.
• Deep knowledge of enterprise risk management, operational risk, financial risk, compliance risk, regulatory expectations, internal controls, and risk governance practices.
• Strong understanding of how CROs evaluate risk tradeoffs across business strategy, regulatory requirements, capital planning, liquidity, operational resilience, technology, cybersecurity, third-party exposure, and reputational impact.
• Proven experience advising executive teams, boards, regulators, audit committees, or senior stakeholders on risk posture, remediation priorities, emerging risks, and control effectiveness.
• Demonstrated experience in developing or evaluating risk policies, risk appetite frameworks, key risk indicators, stress scenarios, risk taxonomies, model governance processes, or remediation plans.
• Bachelor’s degree in finance, economics, accounting, risk management, business, law, engineering, data science, or a related field; advanced degree or relevant certifications such as FRM, PRM, CPA, CFA, CRM, CRISC, CISA, or equivalent professional credentials are preferred.
• Previous experience with AI data training, annotation, or evaluating AI-generated technical content is a strong plus.