Senior Azure Data Engineer

Location : Oshawa, ON (hybrid)

Job Overview

The Senior Azure Data Engineer is responsible for designing, building, and supporting scalable, secure, and high-performance data platforms that enable enterprise analytics, reporting, and data-driven digital services. Working within a cross-functional Agile delivery team, this role develops and operates modern data lakehouse and data warehouse solutions using Microsoft Azure and Databricks, ensuring data products are reliable, well-governed, and optimized for downstream consumption.

Key Responsibilities

Design, develop, and maintain modular, scalable ELT / ETL data pipelines using Azure Data Factory, Azure Databricks, and related Azure services.

Build and support data lake, lakehouse, and data warehouse solutions that meet enterprise analytics, reporting, and application needs.

Implement curated, business-centric data models in collaboration with Data Modelers and Data Architects to provide trusted, reusable datasets.

Develop optimized data processing frameworks using Python, PySpark, Spark SQL, and SQL to handle large-scale, structured and semi-structured datasets.

Clean, transform, and optimize datasets for performance, applying data quality, lineage, and governance controls throughout the data lifecycle.

Collaborate closely with Business Analysts, Data Analysts, Data Scientists, Solution Architects, and Senior Data Engineers to deliver integrated data products.

Integrate data pipelines with CI / CD and DevOps processes to support automated deployment, versioning, and release management.

Implement and manage orchestration and scheduling of data pipelines to meet latency, dependency, and availability requirements.

Monitor and support production data solutions, providing Tier 2 support, troubleshooting ingestion, transformation, and performance issues as needed.

Apply security best practices, including role-based access control, to ensure data is protected in transit and at rest.

Participate in peer code reviews, contribute to shared tooling, and support automation efforts to reduce operational toil.

Maintain accurate technical documentation, including as-built pipelines, data products, and metadata within data catalogs such as Microsoft Purview.

Support Business Intelligence teams by enabling dimensional modeling, aggregation optimization, and high-performance data access for reporting tools such as Power BI.

Work within an Agile SCRUM framework, contributing to backlog refinement, story development, and sprint delivery activities.

Qualifications

Completion of a four-year university degree in Computer Science, Computer or Software Engineering, Data Engineering, Data Analytics, or a related discipline.

Proven experience as a Data Engineer designing, building, and supporting enterprise-scale data pipelines.

Strong hands-on experience with Azure Data Factory, Azure Data Lake Storage, Azure Synapse Analytics, and Azure Databricks.

Demonstrated experience building and supporting data lakehouse and data warehouse architectures.

Advanced proficiency in Python, PySpark, Spark SQL, and SQL.

Solid understanding of data structures, distributed data processing frameworks, and performance optimization techniques.

Working knowledge of data governance, metadata management, and data quality principles.

Experience working in Agile delivery environments.

Strong communication skills, with the ability to explain technical concepts to non-technical stakeholders.

Spirit Omega is committed to a diverse and inclusive workplace. We welcome applications from anyone, including members of Indigenous peoples, Women, visible minorities, persons with disabilities, persons of minority sexual orientations and gender identities, and others with the skills and knowledge to productively engage with diverse communities.

Looking for more opportunities? Check out our website at  jobs.spiritomega.com

#INDSPO