Talent.com
Aqila Systems Inc.
Senior GenAI QA Engineer - TorontoAqila Systems Inc. • Toronto, Canada, United States
Rechercher d'autres offres
Senior GenAI QA Engineer - Toronto

Senior GenAI QA Engineer - Toronto

Aqila Systems Inc. • Toronto, Canada, United States
Il y a 3 heures
Salaire
104 000,00 $CA – 124 800,00 $CA par an
Type de contrat
  • Temps plein
Description de poste

Salary: $104,000 - 124,800 per year

Requirements:

  • 4+ years of experience in software QA engineering, SDET, or AI/ML engineering, with a strong foundation in API testing and automation.
  • 2+ years of hands-on experience evaluating production GenAI/LLM applications, including RAG-based or Agentic AI systems.
  • Strong Python skills for test automation and CI/CD integration.
  • In-depth understanding of AI quality, hallucination risk, guardrail evaluation, and adversarial or red-team testing.
  • Ability to turn conceptual AI evaluation into practical, measurable, repeatable metrics.
  • Experience with or familiarity with Python, AWS Bedrock, Azure OpenAI, Anthropic Claude, LangGraph, LangChain, Semantic Kernel, Promptfoo, Langfuse, and DeepEval.

Responsibilities:

  • Lead quality assurance and evaluation strategy for our Agentic AI platform, with a focus on safety, hallucination risk, and PII/PHI protection.
  • Develop automated evaluation tools in Python and integrate quality checks into CI/CD pipelines.
  • Create practical, repeatable evaluation frameworks and test datasets for non-deterministic AI systems.
  • Test the effectiveness of AI guardrails using functional, adversarial, and red-team approaches.
  • Own the core software QA foundation for APIs, backend services, and user-facing workflows.

Technologies:

  • Agentic AI
  • AI
  • AI Agents
  • API
  • AWS
  • Azure
  • Backend
  • CI/CD
  • Support
  • LLM
  • Langfuse
  • Python
  • RAG
  • Cloud

More:

We are seeking a Senior GenAI QA Engineer for a live, production-grade Agentic AI platform in the healthcare domain. This hands-on, high-impact role involves close collaboration with senior AI engineers to support the safety and reliability of our evolving AI agents. The position is a three-month contract with a strong possibility of extension or conversion to hire. It is remote in Canada, with occasional on-site work in downtown Toronto; the listed work arrangement is hybrid remote in Toronto, Ontario. Compensation is $50–$60 per hour, with expected hours capped at 40 per week.

last updated 47 week of 2026

Créer une alerte emploi pour cette recherche

Senior GenAI QA Engineer - Toronto • Toronto, Canada, United States