Talent.com
Astra North Infoteck Inc.
Senior Quality Engineer – Test Automation, AI Testing & Java/PythonAstra North Infoteck Inc. • Toronto, ON, ca
Senior Quality Engineer – Test Automation, AI Testing & Java/Python

Senior Quality Engineer – Test Automation, AI Testing & Java/Python

Astra North Infoteck Inc. • Toronto, ON, ca
15 hours ago
Job type
  • Full-time
Job description

  1. Quality Engineer – Test Automation, Java/Python & GenAI

    Roles and Responsibilities:

    • Partner with DevOps, Development, Product, and QE teams to design and execute exploratory, automated, and AI-focused tests that ensure product quality.
    • Define quality strategies, test plans, test cases, and test data for functional, integration, system, regression, API, database, and acceptance testing.
    • Design, develop, and maintain scalable test automation frameworks using Java, Python, or Groovy.
    • Build automated tests for AI-enabled features, including prompt behavior, response accuracy, hallucination risks, safety, reliability, latency, and regression coverage.
    • Validate AI integrations, APIs, microservices, data pipelines, model interactions, and third-party services.
    • Establish testing approaches for both deterministic and non-deterministic AI behavior, including evaluation datasets, expected behavior, quality thresholds, and repeatability.
    • Integrate automated testing and AI quality checks into CI/CD pipelines in partnership with DevOps teams.
    • Use tools and frameworks such as Selenium, Cucumber/Gherkin, Playwright, RestAssured, JUnit, TestNG, ReadyAPI, and JMeter.
    • Identify, document, prioritize, and track defects through resolution.
    • Analyze test coverage, quality trends, AI evaluation results, and production feedback to identify gaps and drive continuous improvement.
    • Collaborate with cross-functional Agile teams to deliver reliable, secure, and user-focused product features.
    • Communicate test results, risks, quality metrics, and release recommendations clearly to technical and non-technical stakeholders.

    Required Skills:

    • 5+ years of relevant experience in Quality Engineering, Quality Assurance, test automation, or software development.
    • 5+ years of programming experience with Java, Python, Groovy, or a comparable language.
    • 5+ years of experience building and maintaining test automation frameworks.
    • Hands-on experience with Selenium, Cucumber/Gherkin, Playwright, RestAssured, or similar automation technologies.
    • Experience testing functional and non-functional requirements, APIs, integrations, databases, microservices, and regression suites.
    • Experience with testing frameworks and tools such as JUnit, TestNG, qTest, ReadyAPI, and JMeter.
    • Understanding of TDD, ATDD, BDD, continuous testing, and risk-based testing practices.
    • Experience testing AI or machine learning-powered applications, APIs, conversational systems, or generative AI features.
    • Understanding of AI quality concepts such as prompt evaluation, grounding, accuracy, relevance, toxicity, bias, hallucination detection, and response consistency.
    • Ability to create effective test data and evaluation datasets for AI-driven functionality.
    • Strong analytical, problem-solving, communication, and technical documentation skills.
    • Ability to work effectively in a fast-paced, cross-functional Agile environment.

    Nice to Have:

    • Experience with AI evaluation frameworks, LLM testing, prompt engineering, retrieval-augmented generation, embeddings, or model observability.
    • Experience with Git, Bitbucket, Jenkins, CI/CD pipelines, Artifactory, SonarQube, Jira, Confluence, AWS, Kubernetes, Maven, or Gradle.
    • Experience testing cloud-native applications and distributed microservice architectures.
    • Experience with security, performance, accessibility, or reliability testing.
    • Experience developing automation scripts and reusable testing utilities.
    • Experience with Tableau or other analytics and visualization platforms.
    • CSTE certification or an equivalent quality engineering certification.
    • Experience working directly with DevOps teams to implement automated quality gates and release controls.






Create a job alert for this search

Senior Quality Engineer – Test Automation, AI Testing & Java/Python • Toronto, ON, ca

Similar jobs

Senior Quality Engineer - Automation & Shift-Left

LuxoftToronto, ON, CA
Full-time

A leading technology firm is looking for a Senior Quality Engineer to enhance the quality standards within the Real Asset Technology team.This position centers around implementing test strategies, ... Show more

 • Promoted

Senior QE Engineer - Onsite Role

Aarorn Technologies IncToronto, ON, CA
Full-time

Transform data engineering as a Senior QE Engineer in Toronto, ON.This role requires four days onsite per week and focuses on developing automated testing solutions and ensuring data integrity.In t... Show more

 • Promoted

Python Test Automation Engineer + Selenium

Veracity Software IncToronto, ON, CA
Full-time +1

Python Test Automation Engineer + Selenium.Python Test Automation Engineer + Selenium.Quality Assurance, Manual Testing, and Software Testing skills.Ability to create effective test cases.Strong an... Show more

 • Promoted

Senior Automation Engineer

TubiToronto, ON, CA
Full-time

We are looking for a talented Automation Engineer to join our Automation Engineering team in Toronto.In this role, you will be responsible for designing and implementing automated tests for mobile ... Show more

 • Promoted

Senior Automation Leader in Test Frameworks

Pacer GroupToronto, ON, CA
Full-time

Become a key player at Pacer Group as a Senior Automation Lead, bringing over a decade of experience in Quality Engineering and Test Automation within enterprise settings.This role demands mastery ... Show more

 • Promoted

Senior Qa Automation Engineer (Canada)

SoundHound Inc.Toronto, Canada
Full-time

SoundHound AI believes every person should be able to interact naturally with the products around them–by simply talking.With a global reach spanning two dozen languages, we build Voice AI products... Show more

 • Promoted

Senior QE/SDET: Java/Python Automation & API Testing

Iris Software Inc.Toronto, ON, CA
Full-time

A software solutions provider is seeking a QE/SDET in Toronto, ON for a hybrid position (4 days a week).The role requires 8-10+ years of experience, strong programming skills in Java/Python, and ex... Show more

 • Promoted

Senior QA Engineer: QMS & Test Automation Leader

RxPxToronto, ON, CA
Full-time

A global digital health platform is seeking an experienced Senior Quality Engineer to join their team in Canada.The role focuses on maintaining quality management systems (QMS), driving compliance ... Show more

 • Promoted

Senior Software Development Engineer in Test

Magnet ForensicsToronto, ON, CA
Full-time

We are seeking a highly skilled Senior Software Development Engineer in Test (SDET) to support the development and deployment of next‑generation AI‑powered products.The role involves architecting, ... Show more

 • Promoted

Senior Mobile Test Automation Engineer

Princeton IT Services, IncToronto, ON, CA
Full-time

Become a Senior Mobile Test Automation Engineer, focusing on mobile application quality, either in Toronto or remote.Leverage automation to ensure exceptional performance and reliability.We are loo... Show more

 • Promoted

Senior Qa Engineer Automation Banking Solutions

Quantum World Technologies Inc.Toronto, Canada
Full-time

Become a vital part of our banking solutions team as a Senior QA Automation Engineer.Focus on creating automated solutions to verify banking applications and services.With extensive experience in Q... Show more

 • Promoted

Senior QA Engineer Automation Banking Solutions

Quantum World Technologies Inc.Toronto, Ontario, Canada
Full-time

Become a vital part of our banking solutions team as a Senior QA Automation Engineer.Focus on creating automated solutions to verify banking applications and services.With extensive experience in Q... Show more

 • Promoted

Senior Software Engineer for AI Solutions

Thomson ReutersToronto, Ontario, Canada
Full-time

Elevate your career as a Senior Software Engineer at Thomson Reuters, focusing on AI systems that transform accounting practices.This hybrid opportunity combines backend development with innovative... Show more

 • Promoted

Senior Software Development Engineer in Test (SDET) - AI Cluster

Cerebras Systems, Inc.Toronto, ON, CA
Full-time

Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs.This architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than... Show more

 • Promoted

Senior Software Development Engineer in Test

KongToronto, ON, CA
Full-time

Are you ready to unlock intelligence?.If you don’t think you meet all of the criteria below but are still interested in the job, please apply.Nobody checks every box - we’re looking for candidates ... Show more

 • Promoted

Senior Qa Automation Engineer – Elevate Enterprise Testing

CAAT Pension PlanToronto, Canada
Full-time

A leading pension provider in Ontario is seeking a QA Automation Engineer to design and develop automated test strategies, ensuring quality across business applications.The ideal candidate will hav... Show more

 • Promoted

Quality Engineer - Python & Automation at TCS

Tata Consultancy ServicesToronto, ON, CA
Full-time

Join Tata Consultancy Services as a Quality Engineer specializing in Python and automation.Focus on implementing valuable ETL testing solutions to ensure high-quality software outcomes.TCS is looki... Show more

 • Promoted

Senior QA Automation Engineer (Canada)

SoundHound Inc.Toronto, ON, CA
Full-time

SoundHound AI believes every person should be able to interact naturally with the products around them–by simply talking.With a global reach spanning two dozen languages, we build Voice AI products... Show more

 • Promoted

Senior QA Automation Engineer: Lead Test Automation

Rubicon PathToronto, ON, CA
Full-time

A technology solutions firm in Toronto is looking for a Senior Automation Developer with extensive experience in automation testing.The role requires identifying automation opportunities, executing... Show more

 • Promoted

Senior QA Automation Engineer – Elevate Enterprise Testing

CAAT Pension PlanToronto
Full-time

A leading pension provider in Ontario is seeking a QA Automation Engineer to design and develop automated test strategies, ensuring quality across business applications.The ideal candidate will hav... Show more