Talent.com
Advanced Micro Devices, Inc
Principal Software Developer – AI/ML Performance Validation & Systems TestingAdvanced Micro Devices, Inc • MARKHAM, Ontario, Canada
Principal Software Developer – AI/ML Performance Validation & Systems Testing

Principal Software Developer – AI/ML Performance Validation & Systems Testing

Advanced Micro Devices, Inc • MARKHAM, Ontario, Canada
1 day ago
Job type
  • Full-time
Job description

ADVANCE YOUR CAREER. ADVANCE THE WORLD.

At AMD, we believe technology has the power to solve the world’s most important challenges. From advancing healthcare and scientific discovery to powering AI and the technologies people rely on every day, innovation at AMD is shaping the future.

Whether you’re designing next-gen processors, enabling AI breakthroughs, or bringing leading edge products to market, every role at AMD contributes to something bigger — technology that moves the world forward. Join us and, together, we’ll advance your career.

THE ROLE:

We are seeking a Principal Software Engineer to serve as the senior technical leader for ROCm software validation across compute workloads and server-class systems. In this individual-contributor leadership role, you will define how AMD proves ROCm is ready to ship — from unit and component testing, through full-stack workload validation, to multi-node system-level qualification on AMD Instinct™ GPU platforms. You will set the technical direction for validation strategy, build and evolve the test infrastructure that gates every ROCm release, and personally drive the hardest debugging, characterization, and qualification problems. Your work directly determines the quality bar experienced by hyperscalers, OEMs, sovereign-AI customers, and the open-source community running ROCm in production.

THE PERSON:

You will set the technical direction for validation strategy, build and evolve the test infrastructure that gates every ROCm release, and personally drive the hardest debugging, characterization, and qualification problems. Your work directly determines the quality bar experienced by hyperscalers, OEMs, sovereign-AI customers, and the open-source community running ROCm in production.

KEY RESPONSIBILITIES:

  • Own the end-to-end validation architecture for ROCm — unit, integration, framework, workload, performance, stress, stability, scale-out, and system-level test layers — across multiple GPU generations and server platforms.
  • Define release-qualification gates and exit criteria for ROCm software releases (functional coverage, performance regressions, stability hours, scale targets, RAS criteria) and drive the org to meet them.
  • Architect the test infrastructure — distributed test runners, GitHub Actions / Jenkins / internal CI fleets, hardware lab orchestration, result data lakes, flaky-test detection, bisection automation, and self-service developer pre-submit pipelines.
  • Champion modern, agile quality engineering — shift-left testing, test pyramids, contract testing between layers, hermetic test environments, deterministic reproducers, and continuous validation in trunk.
  • Set the bar for GitHub-based quality workflows — PR gating policy, required checks, code-coverage standards, bug-bash and triage cadences, and disciplined issue management across ROCm/* repositories and partner upstream projects.
  • Lead complex escalation debug — partner with development, hardware, firmware, and customer-facing teams to root-cause the hardest multi-day, multi-node, multi-component failures and convert findings into durable test coverage.
  • Influence the roadmap — work with product management, silicon, platform, and software architecture to ensure validation readiness for next-generation Instinct GPUs and server platforms before tape-in milestones and silicon arrival.
  • Mentor and elevate Senior and Staff validation engineers, SDETs, and SQA leads; raise the technical bar through design review, code review, and written guidance.
  • Represent ROCm validation externally — strategic customer engagements, OEM qualification programs, and open-source community quality initiatives.
  • Lead system-level testing for server nodes — multi-GPU topologies, PCIe/Infinity Fabric/xGMI, BMC/IPMI, thermal/power, firmware interactions, and multi-node fabric (Ethernet/InfiniBand/UALink) bring-up and validation.Drive compute workload validation and characterization — LLM training and inference (PyTorch, vLLM, Triton, JAX), recommender systems, scientific HPC kernels, MLPerf-class benchmarks — establishing reproducible methodology, baselines, and regression tracking.

PREFERRED EXPERIENCE:

  • Software engineering experience in validation, SDET, or quality engineering, including experience leading complex systems validation.
  • Expert Python for test automation and infrastructure; strong C++ for debugging and extending production code.
  • Deep validation expertise in two or more of the following:GPU software stacks (ROCm, CUDA, oneAPI, SYCL)AI/ML frameworks (PyTorch, TensorFlow, JAX, Triton, vLLM)HPC runtimes and communication libraries (MPI, RCCL/NCCL, UCX, Libfabric)Linux kernel, GPU drivers, or accelerator firmwareDistributed systems and large-scale cluster software
  • Experience validating multi-GPU, multi-node server platforms, including stress, soak, fault injection, and RAS testing.
  • Experience defining and delivering release qualification programs for hyperscalers, OEMs, or Tier-1 customers.
  • Contributions to validation, CI, or test infrastructure for ROCm, PyTorch, LLVM, Triton, vLLM, or similar open-source projects.
  • Experience leading adoption of agentic AI workflows, including automated testing, AI-driven debugging, MCP, and RAG-based engineering solutions.
  • Experience validating or operating large-scale GPU clusters (256+ GPUs), including fabric bring-up, health monitoring, and diagnostics.
  • Familiarity with AI training, inference, and HPC benchmark methodologies.
  • Experience with performance validation, profiling tools (rocprof, Omniperf, Nsight), and regression analysis.
  • Familiarity with hardware lab automation, including BMC/IPMI/Redfish, PDU control, serial consoles, automated re-imaging, and topology-aware scheduling.
  • Experience supporting validation for pre-silicon, emulation, and first-silicon accelerator bring-up.

ACADEMIC CREDENTIALS:

  • BS/MS/PhD in Computer Science, Computer Engineering, or related discipline (or equivalent demonstrated experience).

LOCATION: San Jose, California

#LI-DR1

#LI-HYBRID

Benefits offered are described: AMD benefits at a glance.

AMD does not accept unsolicited resumes from headhunters, recruitment agencies, or fee-based recruitment services. AMD and its subsidiaries are equal opportunity, inclusive employers and will consider all applicants without regard to age, ancestry, color, marital status, medical condition, mental or physical disability, national origin, race, religion, political and/or third-party affiliation, sex, pregnancy, sexual orientation, gender identity, military or veteran status, or any other characteristic protected by law. We encourage applications from all qualified candidates and will accommodate applicants’ needs under the respective laws throughout all stages of the recruitment and selection process.

AMD may use Artificial Intelligence to help screen, assess or select applicants for this position. AMD’s “Responsible AI Policy” is available here.

This posting is for an existing vacancy.

THE ROLE:

We are seeking a Principal Software Engineer to serve as the senior technical leader for ROCm software validation across compute workloads and server-class systems. In this individual-contributor leadership role, you will define how AMD proves ROCm is ready to ship — from unit and component testing, through full-stack workload validation, to multi-node system-level qualification on AMD Instinct™ GPU platforms. You will set the technical direction for validation strategy, build and evolve the test infrastructure that gates every ROCm release, and personally drive the hardest debugging, characterization, and qualification problems. Your work directly determines the quality bar experienced by hyperscalers, OEMs, sovereign-AI customers, and the open-source community running ROCm in production.

THE PERSON:

You will set the technical direction for validation strategy, build and evolve the test infrastructure that gates every ROCm release, and personally drive the hardest debugging, characterization, and qualification problems. Your work directly determines the quality bar experienced by hyperscalers, OEMs, sovereign-AI customers, and the open-source community running ROCm in production.

KEY RESPONSIBILITIES:

  • Own the end-to-end validation architecture for ROCm — unit, integration, framework, workload, performance, stress, stability, scale-out, and system-level test layers — across multiple GPU generations and server platforms.
  • Define release-qualification gates and exit criteria for ROCm software releases (functional coverage, performance regressions, stability hours, scale targets, RAS criteria) and drive the org to meet them.
  • Architect the test infrastructure — distributed test runners, GitHub Actions / Jenkins / internal CI fleets, hardware lab orchestration, result data lakes, flaky-test detection, bisection automation, and self-service developer pre-submit pipelines.
  • Champion modern, agile quality engineering — shift-left testing, test pyramids, contract testing between layers, hermetic test environments, deterministic reproducers, and continuous validation in trunk.
  • Set the bar for GitHub-based quality workflows — PR gating policy, required checks, code-coverage standards, bug-bash and triage cadences, and disciplined issue management across ROCm/* repositories and partner upstream projects.
  • Lead complex escalation debug — partner with development, hardware, firmware, and customer-facing teams to root-cause the hardest multi-day, multi-node, multi-component failures and convert findings into durable test coverage.
  • Influence the roadmap — work with product management, silicon, platform, and software architecture to ensure validation readiness for next-generation Instinct GPUs and server platforms before tape-in milestones and silicon arrival.
  • Mentor and elevate Senior and Staff validation engineers, SDETs, and SQA leads; raise the technical bar through design review, code review, and written guidance.
  • Represent ROCm validation externally — strategic customer engagements, OEM qualification programs, and open-source community quality initiatives.
  • Lead system-level testing for server nodes — multi-GPU topologies, PCIe/Infinity Fabric/xGMI, BMC/IPMI, thermal/power, firmware interactions, and multi-node fabric (Ethernet/InfiniBand/UALink) bring-up and validation.Drive compute workload validation and characterization — LLM training and inference (PyTorch, vLLM, Triton, JAX), recommender systems, scientific HPC kernels, MLPerf-class benchmarks — establishing reproducible methodology, baselines, and regression tracking.

PREFERRED EXPERIENCE:

  • Software engineering experience in validation, SDET, or quality engineering, including experience leading complex systems validation.
  • Expert Python for test automation and infrastructure; strong C++ for debugging and extending production code.
  • Deep validation expertise in two or more of the following:GPU software stacks (ROCm, CUDA, oneAPI, SYCL)AI/ML frameworks (PyTorch, TensorFlow, JAX, Triton, vLLM)HPC runtimes and communication libraries (MPI, RCCL/NCCL, UCX, Libfabric)Linux kernel, GPU drivers, or accelerator firmwareDistributed systems and large-scale cluster software
  • Experience validating multi-GPU, multi-node server platforms, including stress, soak, fault injection, and RAS testing.
  • Experience defining and delivering release qualification programs for hyperscalers, OEMs, or Tier-1 customers.
  • Contributions to validation, CI, or test infrastructure for ROCm, PyTorch, LLVM, Triton, vLLM, or similar open-source projects.
  • Experience leading adoption of agentic AI workflows, including automated testing, AI-driven debugging, MCP, and RAG-based engineering solutions.
  • Experience validating or operating large-scale GPU clusters (256+ GPUs), including fabric bring-up, health monitoring, and diagnostics.
  • Familiarity with AI training, inference, and HPC benchmark methodologies.
  • Experience with performance validation, profiling tools (rocprof, Omniperf, Nsight), and regression analysis.
  • Familiarity with hardware lab automation, including BMC/IPMI/Redfish, PDU control, serial consoles, automated re-imaging, and topology-aware scheduling.
  • Experience supporting validation for pre-silicon, emulation, and first-silicon accelerator bring-up.

ACADEMIC CREDENTIALS:

  • BS/MS/PhD in Computer Science, Computer Engineering, or related discipline (or equivalent demonstrated experience).

LOCATION: San Jose, California

#LI-DR1

#LI-HYBRID

Benefits offered are described: AMD benefits at a glance.

AMD does not accept unsolicited resumes from headhunters, recruitment agencies, or fee-based recruitment services. AMD and its subsidiaries are equal opportunity, inclusive employers and will consider all applicants without regard to age, ancestry, color, marital status, medical condition, mental or physical disability, national origin, race, religion, political and/or third-party affiliation, sex, pregnancy, sexual orientation, gender identity, military or veteran status, or any other characteristic protected by law. We encourage applications from all qualified candidates and will accommodate applicants’ needs under the respective laws throughout all stages of the recruitment and selection process.

AMD may use Artificial Intelligence to help screen, assess or select applicants for this position. AMD’s “Responsible AI Policy” is available here.

This posting is for an existing vacancy.

Create a job alert for this search

Principal Software Developer – AI/ML Performance Validation & Systems Testing • MARKHAM, Ontario, Canada

Similar jobs

Hybrid Principal Enterprise Architect - AI Focus

Priceline.com LLCToronto, Ontario, Canada
Full-time

Take on a leadership role as a Principal Enterprise Architect focusing on AI Platforms with a flexible hybrid work model.Drive the technical vision for AI and Machine Learning systems.This strategi... Show more

 • Promoted

Senior AI/ML Applications Architect

GE VernovaMarkham, York region, Canada
Full-time

GE Vernova is accelerating the path to more reliable, affordable, and sustainable energy, while helping our customers power economies and deliver the electricity that is vital to health, safety, se... Show more

 • Promoted

Principal Software Architect — AI-Driven Data Platforms Lead

AlphaSenseToronto, Ontario, Canada
Full-time

A leading market intelligence firm in Canada seeks an experienced engineer to own the architecture and evolution of a large-scale data extraction platform.The role involves designing reliable syste... Show more

 • Promoted

Ai Solutions Architect (Senior And Principal)

RAVL Inc.Toronto, Canada
Permanent

RAVL helps technologists accelerate their careers.At RAVL, we connect strategy with execution, care deeply about the people we work with, and measure success by the lasting impact we leave behind.O... Show more

 • Promoted

Senior Developer (Ai/Ml/Gen Ai Solutions) - C$104,000 - C$154,000 A Year

TelusToronto County, Canada
Full-time

Seeking a Senior Full Stack Developer with AI/ML/Gen AI expertise to lead cross-functional teams in designing and implementing intelligent applications, mentor junior developers, and drive technica... Show more

 • Promoted

Senior Ml Engineer - Ai Platform & Ranking Systems - C$160,000 - C$220,000 A Year

Relationship Intelligence PlatformEast York, Canada
Full-time

Seeking a Senior Machine Learning Engineer to lead ML engineering efforts, focusing on the ML lifecycle and building recommendation systems.Competitive salary and benefits offered. Show more

 • Promoted

Remote Ai/Ml Software Engineer - C$62,000 - C$75,000 A Year

Terra Remote SensingEast York, Canada
Remote
Full-time

Intermediate programmer needed for AI/ML projects and system enhancements in a geospatial data firm.Requires a degree and 3-5 years of programming experience. Show more

 • Promoted

Ai Performance Architect: Hardware-Software Co-Design (Remote) - $100,000 - $300,000 A Year - Remote

TenstorrentEast York, Canada
Remote
Full-time

AI Performance Architect sought to model and optimize AI workloads.Requires C++ and Python experience.Salary ranges from $100k to $500k. Show more

 • Promoted

Senior Ml Engineer - Ai Platform & Ranking Systems - C$160,000 - C$220,000 A Year

Affinity.coToronto, Canada
Full-time

A leading relationship intelligence platform in Toronto is seeking a Senior Machine Learning Engineer to advance their ML engineering capabilities.This role involves owning the ML lifecycle and tra... Show more

 • Promoted

AI/ML Lead (Senior Machine Learning Engineer – Full-time Leadership Role)

Aurelian Venture AIToronto, Ontario, Canada
Full-time

AI/ML Lead (Senior Machine Learning Engineer – Full-time Leadership Role).Full-Time, Contract, Hands‑on Technical Leadership.CAD $180,000 – $250,000 base + significant equity + performance bonuses ... Show more

 • Promoted

Senior Applied Ai Systems Developer / Architect - C$120,000 - C$150,000 A Year

Klick GroupToronto, Canada
Full-time

Senior developer/architect to lead AI-enabled features for the Genome platform, focusing on scalable systems and integrating AI/LLM capabilities. Show more

 • Promoted

Senior Applied Ai Engineer — Product & Revenue Systems

QEA TechMarkham, Canada
Full-time

Build and deploy AI systems that directly drive revenue by embedding AI across QEA’s product, sales, and marketing workflows.This role focuses on execution: turning QEA’s building data, imagery, an... Show more

 • Promoted

Senior Ml Engineer - Ai Platform & Ranking Systems - C$160,000 - C$220,000 A Year

J-18808-LjbffrToronto, Canada
Full-time

A leading relationship intelligence platform in Toronto is seeking a Senior Machine Learning Engineer to advance their ML engineering capabilities.This role involves owning the ML lifecycle and tra... Show more

 • Promoted

AI Platform Software Engineer

GeotabToronto, Ontario, Canada
Full-time

Empower AI initiatives as a Senior Data Platform Developer focused on machine learning technologies.Develop, maintain, and enhance AI applications while ensuring the performance and reliability of ... Show more

 • Promoted

Senior AI Developer: Agentic Systems & Platform Lead

Constellation Dealer GroupToronto, Ontario, Canada
Full-time

A North American software provider is seeking a Senior Developer to design and build AI-native systems that impact business outcomes.The role emphasizes collaborative system design and engineering ... Show more

 • Promoted

Senior AI Software Engineer - On-Device ML (C++17)

QualcommMarkham, York region, Canada
Full-time

A leading technology company in York Region, Markham is seeking a software engineer to develop AI solutions using modern C++17.You will work on cutting-edge technology for Qualcomm Hexagon Processo... Show more

 • Promoted

Remote AI & ML Developer — LLMs, RAG & APIs

OrganimiToronto, ON, CA
Remote
Full-time

A leading B2B SaaS company is seeking an experienced AI / Machine Learning Developer to work on innovative AI models and systems.The candidate will train and deploy large language models and relate... Show more

 • Promoted

Ai-Ml Engineer

ampliphi RMS softwareToronto, Canada
Full-time

Global Talent Acquisition at Valsoft Corporation and Aspire SoftwareWe are hiring for an AI-ML Engineer to join our growing team at Ampliphi in Toronto!As an AI Engineer at Ampliphi, you will be at... Show more