Talent.com
United Software Group Inc
QA Engineer (AI)United Software Group Inc • Montreal, QC, Canada
QA Engineer (AI)

QA Engineer (AI)

United Software Group Inc • Montreal, QC, Canada
6 days ago
Job type
  • Full-time
  • Quick Apply
Job description

QA Engineer (AI)

Location: Montreal, QC (4 Days Onsite/Week)
Interview: Face-to-Face Required
Employment Type: Contract

Job Summary

We are looking for an experienced QA Engineer (AI) to lead testing initiatives for AI/LLM-powered applications. The ideal candidate will have expertise in Generative AI testing, RAG pipelines, prompt engineering, automation, and API testing, along with strong experience in QA automation and enterprise application testing.

Key Responsibilities

  • Design and execute comprehensive test strategies for AI/LLM applications.
  • Validate prompt quality, hallucination detection, bias, safety, and response accuracy.
  • Test Retrieval-Augmented Generation (RAG) pipelines, vector databases, embeddings, and retrieval accuracy.
  • Build and maintain automated test frameworks using Python and Java.
  • Perform API, integration, regression, functional, and performance testing.
  • Validate LangChain/LangGraph workflows and AI agent integrations.
  • Collaborate with developers, product owners, and business stakeholders.
  • Lead defect triage, quality reporting, and production support.
  • Mentor QA team members and promote QA best practices.

Required Skills

  • 7+ years of QA/Quality Engineering experience.
  • 3+ years in a Senior or Lead QA role.
  • Strong experience with:
    • AI/LLM Testing
    • Prompt Engineering
    • Hallucination & Bias Testing
    • RAG Testing
    • Vector Databases
    • LangChain / LangGraph
    • Python & Java
    • API Automation
    • Selenium, Playwright, Robot Framework, or similar
    • CI/CD (Jenkins, GitHub, etc.)
  • Experience with SQL, database validation, and reconciliation testing.
  • Knowledge of LLM architecture, embeddings, tokenization, and inference.

Preferred Skills

  • Model Context Protocol (MCP).
  • AI Safety, Red Teaming, and Responsible AI testing.
  • Capital Markets or Securitization domain experience.
  • DevOps and cloud-based testing.
  • Agile/Scrum methodologies.

eye

Create a job alert for this search

QA Engineer (AI) • Montreal, QC, Canada

Similar jobs

AI Engineer

BioRenderMontreal (administrative region), QC, CA
Full-time

At BioRender, we don’t settle for average, and neither should you.We’re building tools that transform how scientists communicate complex ideas, and we need someone who’s ready to turn ambitious ide... Show more

 • Promoted

Expert Senior AI Engineer - Remote Role

NTT DATA, Inc.Montreal (administrative region), QC, CA
Remote
Full-time

Make a difference in AI technology with NTT DATA as a Remote Senior AI Engineer from Toronto, Ontario.Focus on designing AI models and enhancing existing architectures for impactful solutions.As a ... Show more

 • Promoted

Solutions Engineer for Gatekeeper's AI Platform

GatekeeperMontreal (administrative region), QC, CA
Full-time

Join Gatekeeper to empower compliance-driven organizations as a Solutions Engineer specializing in customer success.This fully remote role allows you to help clients leverage innovative AI features... Show more

 • Promoted

Senior QA Automation Engineer – 100% Remote *Completed*

Corporate Recruiters Ltd.Montreal (administrative region), QC, CA
Remote
Temporary

Our client develops advanced (including AI) based software for scheduling complex operations.They offer an entrepreneurial and dynamic environment that emphasizes delivering innovative and quality ... Show more

 • Promoted

Remote Kotlin QA Engineer & AI Trainer (Freelance)

MindriftMontreal (administrative region), QC, CA
Remote
Full-time

A forward-thinking tech company seeks a Freelance Software Developer skilled in Kotlin and iOS development.This remote role offers flexibility, with responsibilities including designing automated t... Show more

 • Promoted

Senior QA Infrastructure Engineer - Remote AI Testing

StackAdapt Inc.Montreal (administrative region), QC, CA
Remote
Full-time

A leading marketing technology firm is seeking a Senior Quality Engineer to lead the design and implementation of testing infrastructure projects in a remote-first environment.The ideal candidate w... Show more

 • Promoted

QA Automation Engineer - Trading Platform

Evolvic Inc.Montreal (administrative region), QC, CA
Temporary

A leading software development company in Montreal seeks a QA Analyst to enhance software quality for the Orchestrade application.The role involves executing test campaigns, improving test processe... Show more

 • Promoted

AI Enablement Engineer - AI Tools & Agents (Equity)

BasetenMontreal
Full-time

A leading AI platform company in Montreal is looking for an AI Enablement Engineer.This role involves building and integrating AI tools that enhance productivity for engineering workflows.The ideal... Show more

 • Promoted

Senior QA Automation Engineer - Web & Mobile

GreenShieldMontreal (administrative region), QC, CA
Full-time

A not-for-profit health benefits provider in Montreal is seeking a highly skilled QA Automation Engineer III.The role involves designing and maintaining test automation frameworks for web and mobil... Show more

 • Promoted

Forward Deployed AI Engineer, Canada - BCG X

BCG XMontreal (administrative region), QC, CA
Permanent

Boston Consulting Group partners with leaders in business and society to tackle their most important challenges and capture their greatest opportunities.BCG was the pioneer in business strategy whe... Show more

 • Promoted

Contract Software Engineer Specializing in Advanced AI Techniques

TuringMontreal (administrative region), QC, CA
Full-time

Embrace an exciting challenge as a Software Engineer specializing in AI.Collaborate with teams in a remote setting to elevate AI code quality and efficiency.You will be responsible for applying you... Show more

 • Promoted

Senior/ Lead - AI Engineer

FICOsaint-esprit, qc, ca
Full-time

As a Senior Engineer on our Applied AI team, you will be at the forefront of building AI-powered software that transforms how our platform operates.You will design, build, and maintain production-g... Show more

 • Promoted

Applied AI Engineer

Simpro SoftwareMontreal (administrative region), QC, CA
Full-time

Le Applied AI Software Engineer sera responsable de la conception technique rapide et de la livraison d'agents et de cadres d'IA construits sur les SIMPRO FSM.En étroite collaboration avec le dépar... Show more

 • Promoted

QA Automation Engineer - Pre-Trade Capital Markets

QUANTEAM - North America (RAINBOW PARTNERS Group)Montreal
Full-time

A leading financial technology firm in Montreal is seeking an Associate to manage the testing and support of Sales applications used in capital markets.The ideal candidate will have strong skills i... Show more

 • Promoted

Innovative AI Engineer – Pulsora Canada

PulsoraMontreal (administrative region), QC, CA
Full-time

Leverage your AI expertise at Pulsora as a Remote Engineer focused on developing impactful AI applications.This full-time role prioritizes cutting-edge technology for sustainability efforts.As part... Show more

 • Promoted

Lead Software Engineer - AI and SaaS

Human AgencyMontreal (administrative region), QC, CA
Full-time

Take the lead as a Software Engineer, specializing in AI and SaaS application development.Tackle backend and frontend challenges while driving innovative solutions for complex enterprise problems.I... Show more

 • Promoted

Senior ML Engineer, AI Governance – Remote Canada

OptroMontreal (administrative region), QC, CA
Remote
Full-time

A leading SaaS company based in Canada is seeking a Machine Learning Engineer to innovate risk management solutions using cutting-edge AI techniques.Your role involves developing product features, ... Show more

 • Promoted

Dynamic AI Agent Engineer Delivering Real-World Solutions

CrestaMontreal (administrative region), QC, CA
Full-time

Be at the forefront of AI technology by developing innovative agents that drive business efficiency and enhance customer satisfaction.In this role, leverage your skills to implement AI solutions th... Show more

 • Promoted

Applied AI Engineer

Norbert HealthMontreal (administrative region), QC, CA
Full-time

Norbert is building autonomous robots that deliver healthcare.Our AI sensing platform enables existing robotic platforms to become care team members: rounding on patients, capturing vitals without ... Show more

 • Promoted

Remote Senior QA Automation Engineer

RELQ TECHNOLOGIESMontreal (administrative region), QC, CA
Remote
Full-time

A leading technology firm in Canada is seeking a skilled Sr.QA Automation Engineer to design and maintain automated test frameworks.This role requires collaboration with developers and product team... Show more