As an Azure and Databricks Data Engineer, you will be responsible for designing, building and supporting the data driven applications which enable innovative, customer centric digital experiences.
Will be working as part of a cross-discipline agile team who help each other solve problems across all business areas.
Will build reliable, supportable & performant data lake & data warehouse products to meet the organization’s need for data to drive reporting analytics, applications, and innovation.
Will employ best practice in development, security, and accessibility and design to achieve the highest quality of service for our customers.
Build and productionize modular and scalable data ELT / ETL pipelines and data infrastructure leveraging the wide range of data sources across the organization.
Build curated common data models designed by the Data Modelers that offer an integrated, business-centric single source of truth for business intelligence, reporting, and downstream system use, in collaboration with Data Architect.
Work closely with infrastructure, and cyber teams and Senior Data Developers to ensure data is secure in transit and at rest.
Clean, prepare and optimize datasets for performance, ensuring lineage and quality controls are applied throughout the data integration cycle.
Support Business Intelligence Analysts in modelling data for visualization and reporting, using dimensional data modeling and aggregation optimization methods.
Troubleshoot issues related to ingestion, data transformation and pipeline performance, data accuracy and integrity.
Collaborate with Business Analysts, data scientists, Senior Data Engineers, data Data Analysts and, solution Architects and Data Modelers to develop data pipelines to feed our data marketplace.
Assist in identifying, designing, and implementing internal process improvements : automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Work with tools in the Microsoft Stack; Azure Data Factory, Azure Data Lake, Azure SQL Databases, Azure Data Warehouse, Azure Synapse Analytics Services, Azure Databricks, Microsoft Purview, and Power BI.
Work within the agile SCRUM work management framework in delivery of products and services, including contributing to feature & user story backlog item development, and utilizing related Kanban / SCRUM toolsets.
Assist in building data catalog and maintenance of relevant metadata for datasets published for enterprise use.
Develop optimized, performant data pipelines and models at scale using technologies such as Python, Spark and SQL, consuming data sources in XML, CSV, JSON, REST APIs, or other formats.
Document as-built pipelines and data products within the product description, and utilize source control to ensure a maintainable code-base.
Implement orchestration of data pipeline execution designed by Senior Data Engineers to ensure data products meet customer latency expectations, dependencies are managed, and datasets are as up-to-date as possible, with minimal disruption to end-customer use.
Create tooling in collaboration with senior data engineers and data architects to help with day to day tasks, and reduce toil via automation wherever possible.
Work with Continuous Integration / Continuous Delivery and DevOps pipelines to assist in automate infrastructure, code delivery and product enhancement isolation and proper release management and versioning.
Monitor the ongoing operation of in-production solutions, assist in troubleshooting issues, and provide Tier 2 support for datasets produced by the team, on an as-required basis.
Implement and manage appropriate access to data products via role-based access control based on guidance from senior data engineers.
Write and perform automated unit and regression testing for data product builds, assist with user acceptance testing and system integration testing as required, and assist in design of relevant test cases based on guidance from Data Architects.
Participate in peer code review sessions.
Qualifications
Completion of a four-year University education in computer science, computer / software engineering or other relevant programs within data engineering, data analysis, artificial intelligence, or machine learning.
Experience as a Data Engineer designing and building data pipelines.
Fluent in creating data processing frameworks using Python, PySpark, SparkSOL and SOLExperience with Azure Data Factory, ADLS, Synapse Analytics and Databricks
Experience building data pipelines for Data Lakehouses and Data Warehouses
Good understanding of data structures and data processing frameworks
Knowledge of data governance and data quality principles
Effective communication skills to translate technical details to non-technical stakeholders
"Please note, this is a real and current job vacancy. Morson Edge does not use artificial intelligence to screen or select candidates."
Créer une alerte emploi pour cette recherche
25199 Data Engineer • Oshawa
Offres similaires
Data Analytics Engineer
EXL • oshawa, on, ca
Temps plein
Sports Analytics & Engineering Practice, and we’re hiring an.This role focuses on transforming client data into trusted, analytics-ready assets that power dashboards, insights, and decision-making....Voir plus
Dernière mise à jour : il y a 4 jours • Offre sponsorisée
AWS Cloud Engineer
Hays • oshawa, ON, ca
Temps plein
The ideal candidate will help design, implement, and maintain secure, scalable, and compliant AWS environments while driving automation across the organization.
Required Skills & Experience< / ...Voir plus
Dernière mise à jour : il y a 1 jour • Offre sponsorisée
Hands-On FinOps Engineer (Azure) - Python & Data Pipelines
CPUS Engineering Staffing Solutions Inc. • Oshawa
Temps plein
A staffing solutions company is looking for a FinOps Engineer to implement and operate cloud financial management practices focused on Microsoft Azure.
The role requires strong Python coding skills ...Voir plus
Dernière mise à jour : il y a 4 jours • Offre sponsorisée
Field Engineer - Ontario
Kiewit • Oshawa
Temps plein
As a Field Engineer, you will be the primary and lead engineer on the Project, and report directly to the Project Manager.
You will be expected to be a job‑site leader in health and safety, ensuring...Voir plus
Dernière mise à jour : il y a 12 jours • Offre sponsorisée
25-198 - Lead Software Engineer Developer
Morson Talent • Oshawa
Temps plein
The Lead Software Developer / Engineer is a specialist technical role focused on elevating engineering excellence, driving technical strategy, and influencing software development practices for one o...Voir plus
Dernière mise à jour : il y a 2 jours • Offre sponsorisée
DevOps Engineer (GCP)
Fulfillment IQ • Pickering, ON, Canada
Temps plein
Employment Type : Full-Time (Onsite).Reports to : IT Head of the Development team.At Fulfillment IQ, we are transforming the future of logistics technology.
As an award-winning consultancy and technolog...Voir plus
Dernière mise à jour : il y a 18 jours • Offre sponsorisée
Data Analytics Engineer - oshawa
EXL • oshawa, on, ca
Temps plein
Sports Analytics & Engineering Practice, and we’re hiring an.This role focuses on transforming client data into trusted, analytics-ready assets that power dashboards, insights, and decision-making....Voir plus
Dernière mise à jour : il y a 4 jours • Offre sponsorisée
AWS AI Developer - AIDEV 26-00838
NavitasPartners • Oshawa, ON, Canada
Temps plein
Onshore Canada (preferably Greater Toronto Area) – Hybrid (2–3 days onsite).A leading organization is seeking an AWS AI Developer to support a conversational AI solution built on Amazon...Voir plus
Dernière mise à jour : il y a 14 jours • Offre sponsorisée
Senior Analytics Engineer - TekRek
TekRek • oshawa, on, ca
Temps plein
TekRek has partnered with a fast growing firm specializing in AI enablement and data optimization.Our client works with some of the top tier technology firms in Silcom Valley.With rapid growth and ...Voir plus
Dernière mise à jour : il y a 24 jours • Offre sponsorisée
Statistics Private Tutoring Jobs Beaverton
Superprof • Beaverton, Canada
Temps plein +1
Superprof is Canada's #1 tutoring platform, and we're actively recruiting passionate tutors! Whether you're a student, a professional, or simply someone who loves teaching, join the largest communi...Voir plus
Dernière mise à jour : il y a plus de 30 jours • Offre sponsorisée
Azure & Databricks Data Engineer - Hybrid Contract
2iSolutions Inc. • Oshawa
Temps plein
A technology solutions provider is seeking 2 Data Engineers to join their team in Oshawa.The roles are 1-year contracts requiring expertise in Microsoft technologies, particularly Azure and Databri...Voir plus
Dernière mise à jour : il y a 12 jours • Offre sponsorisée
Senior Analytics Engineer
TekRek • oshawa, on, ca
Temps plein
TekRek has partnered with a fast growing firm specializing in AI enablement and data optimization.Our client works with some of the top tier technology firms in Silcom Valley.With rapid growth and ...Voir plus
Dernière mise à jour : il y a 24 jours • Offre sponsorisée
Algebra Private Tutoring Jobs Beaverton
Superprof • Beaverton, Canada
Temps plein +1
Superprof is Canada's #1 tutoring platform, and we're actively recruiting passionate tutors! Whether you're a student, a professional, or simply someone who loves teaching, join the largest communi...Voir plus
Dernière mise à jour : il y a plus de 30 jours • Offre sponsorisée
Personal Support Worker (PSW)
CarePartners • Little Britain, ON, CA
Temps plein
Are you proud to contribute your skills to delivering quality health care? PSWs are an integral part of a health care team, and in this role, you will encourage, support, and promote independence f...Voir plus
Dernière mise à jour : il y a plus de 30 jours • Offre sponsorisée
Geometry Private Tutoring Jobs Beaverton
Superprof • Beaverton, Canada
Temps plein +1
Superprof is Canada's #1 tutoring platform, and we're actively recruiting passionate tutors! Whether you're a student, a professional, or simply someone who loves teaching, join the largest communi...Voir plus
Dernière mise à jour : il y a plus de 30 jours • Offre sponsorisée
Customer Service Agent - 50k-60k / Year - Remote
Spade Recruiting • Brock, Ontario
Télétravail
Temps plein
Quick Apply
We’re looking for enthusiastic, self-driven, individuals to assist existing and prospective clients within our organization.
This position will work with multiple clients throughout the day pr...Voir plus
Dernière mise à jour : il y a 3 jours • Offre sponsorisée
Customer Service - Work from Home - 50k-60k / Year
Spade Recruiting • Brock, Ontario
Télétravail
Temps plein
Quick Apply
We’re looking for enthusiastic, self-driven, individuals to assist existing and prospective clients within our organization.
This position will work with multiple clients throughout the day pr...Voir plus
Dernière mise à jour : il y a 3 jours • Offre sponsorisée
1.0 Core French and Prep Coverage Teacher
Trillium Lakelands District School Board • Woodville, ON, Canada
Temps plein
Trillium Lakelands District School Board Has an Opening for a Long Term.You may apply to any portion of this position.Core French and Prep Coverage Teacher.
Junior, Intermediate and FSL Part 1 requi...Voir plus
Dernière mise à jour : il y a 5 jours • Offre sponsorisée