Devops Jobs in Pickering, ON

Salary: $100,000 - 205,000 per year

Requirements:

• We want 7+ years of software engineering experience, including 5+ years building and deploying AI/ML systems in production.
• We need deep hands-on experience with LLMs, RAG architectures, prompt engineering, structured outputs, tool use, and vector databases such as Pinecone, Qdrant, Weaviate, or pgvector.
• We look for a proven record of choosing between vendor APIs, specialized document AI services, and self-hosted open-source models based on evidence, while managing production cost and latency trade-offs.
• We expect experience building evaluation layers, including golden datasets, regression suites, accuracy targets, and drift monitoring.
• We value fluency with common design patterns and the judgment to know when not to use them, along with experience refactoring REST APIs without breaking consumers.
• We want demonstrated ability to move AI systems from notebook prototypes to scalable, monitored microservices using Docker and AWS.
• We look for experience parsing complex unstructured documents such as PDFs, CAD/BIM tables, and drawings using OCR, multi-modal LLMs, or layout-aware models.
• We expect strong computer science fundamentals, system design expertise, clean coding practices, and solid SQL skills with PostgreSQL or MySQL.
• We need advanced skill in Node.js/TypeScript or Python.
• Nice-to-have experience includes AEC, PropTech, CPQ, supply chain, or technical e-commerce domain knowledge.
• Nice-to-have experience also includes PHP and REST API work, deep learning with PyTorch or TensorFlow, traditional ML methods, user-facing AI features requiring high accuracy and explainability, and mentoring or leading a small engineering team.

Responsibilities:

• We will have you architect and deploy end-to-end AI pipelines for construction technical documents, submittals, schedules, and product cross-referencing.
• We will rely on you to make and defend build-versus-buy decisions for each capability using benchmarks for accuracy, latency, and cost per document.
• We expect you to define and uphold design standards for retrieval and RAG patterns, prompt and schema conventions, evaluation, production monitoring, and provider routing and fallback logic.
• You will serve as a senior technical authority, mentoring engineers and guiding technical decisions across our data and AI platforms.
• You will also identify when a deterministic approach is better than AI and make that case to product and executive stakeholders.
• We will have you build document intelligence pipelines for technical spec sheets, MEP blueprints, equipment schedules, and quotes using OCR, multi-modal models, schema-constrained outputs, validation, and confidence scoring.
• You will develop semantic search, embeddings, and reranking systems to match manufacturer products and part numbers automatically.
• You will help build retrieval and generative workflows within AutoQuote to speed up quoting and improve accuracy, with human review for low-confidence outputs.
• You will own the evaluation strategy for every AI feature, including golden datasets, regression testing, and release gating for model or prompt changes.
• You will set accuracy thresholds with Product and define human review routing and fallback behavior when performance falls below target.
• You will create tooling and benchmarks to compare models and providers using ATSs own documents.
• You will monitor production output for drift and failure patterns and help enforce data governance, quality, and privacy standards.
• You will integrate AI services into our AWS cloud microservices architecture with a focus on availability, latency, and cost efficiency.
• You will own unit economics such as cost per document and per quote, token usage, caching, batching, and vendor rate-limit and failure considerations.
• You will implement guardrails, retries, fallback paths, and continuous learning loops that feed reviewed output back into evaluation data.
• You will collaborate with software and DevOps teams to deliver CI/CD for AI features, with evaluation runs required before deployment.
• You will partner with Product Managers and executive stakeholders to assess feasibility, estimate impact, and shape technical roadmaps for AI initiatives.

Technologies:

• AI
• API
• AWS
• Architect
• BIM
• CI/CD
• Cloud
• DevOps
• Docker
• MySQL
• PHP
• PostgreSQL
• PyTorch
• Python
• RAG
• REST
• SQL
• TensorFlow
• TypeScript
• microservices
• NodeJS
• Embedding Models
• Fine-tuning
• HTTP
• LLM
• MLOps
• Spark
• Web

More:

ATS Software is a dynamic, industrious, innovative, and collaborative company based in Scarborough, Toronto, with this role working in person. We are a new-position employer offering career advancement opportunities, a benefits package for eligible full-time employees that includes medical, vision, and dental coverage, and a culture that supports a healthy work-life balance. Our workplace also includes free parking, foosball, ping pong, a basketball net, gym facilities, and year-round employee events. The role reports to our Director of Technology, and the compensation range is CAD $100,000 to $205,000 plus benefits.

last updated 41 week of 2026