Lead the migration from Glue/Redshift to Databricks
10+ years of professional data engineering experience, with at least 2+ years in a tech-lead or lead engineer capacity.
5-6 years of experience in each of the following areas:
Deep, hands-on experience with Apache Spark (PySpark and Spark SQL), including performance tuning (partitioning, shuffles, skew, caching, file sizing).
Hands-on production experience with Databricks — Delta Lake, Unity Catalog, Workflows, and either Delta Live Tables or a comparable declarative pipeline framework.
Strong production experience with AWS Glue (jobs, crawlers, Data Catalog, triggers) and Amazon Redshift(including Redshift Spectrum, workload management, distribution and sort keys, performance tuning, and Redshift SQL / stored procedures).
Proven experience leading at least one substantial data platform migration or re-platforming effort end-to-end.
Expert SQL skills and strong software-engineering fundamentals: Python, version control, testing, and CI/CD.
Deep knowledge of AWS data services (S3, IAM, KMS, VPC, Lambda, Step Functions, MSK/Kinesis, CloudWatch).
Strong understanding of data modeling for analytics (dimensional, Data Vault, or medallion patterns) and lakehouse architecture.
Excellent communication skills; able to translate between executive stakeholders and engineers, and to write clear design docs and migration plans.
Data Engineer - Minimum 12+ years experience - newmarket • newmarket, on, ca
LatentView Analytics is a leading global analytics and decision sciences provider, delivering solutions that help companies drive digital transformation and use data to gain a competitive advantage... Show more
Client: Enterprise class EMR provider.On behalf of our client, Affinity is seeking a DevOps Engineer to build and evolve the infrastructure behind its flagship EMR platform.You’ll own Kubernetes-ba... Show more
As a Junior Data Engineer, you will join a dynamic data team and contribute to the development, optimization, and maintenance of data pipelines and data infrastructure.You will work closely with da... Show more
Design and implement robust dimensional and relational data models.Own transformation and modeling of curated (Silver/Gold) datasets.Collaborate with analysts, product analytics, data scientists, a... Show more
A leading financial services firm in Toronto is seeking a Junior Data Engineer to contribute to their Centralized Data Platform built on Databricks.The candidate will develop and maintain data pipe... Show more
No C2C or third-party submissions.Workerbee connects workers with employers through trusted introductions.By joining Workerbee you can be matched for project-based, contract, or permanent opportuni... Show more
The once in a century transition to autonomous and electric vehicles is underway and will require a multi‑trillion‑dollar investment in energy and charging infrastructure, and the real estate to si... Show more
Immediate 2-week sprint (up to 40 hours); ongoing part-time starting late May.Could also lead to more projects and hours.Our client is seeking a versatile.You will act as a technical lead, performi... Show more
The Data Engineer will have the responsibility for the design and maintenance of enterprise data warehouse for various projects, as well as providing on-going support for activities impacting all e... Show more
Canada’s fibre, 5G and satellite broadband company for rural living.Xplore is committed to the relentless pursuit of an improved broadband experience for all Canadians.Xplore is building a world‑cl... Show more
BET99 is Canada's Premiere Online Sportsbook and Casino.Launched in 2020, we have consistently innovated the online gaming landscape every step of the way, exponentially growing our customer base a... Show more
Exciting Opportunity in Canada - Remote.Lead the design and implementation/delivery of AI solutions, AI Training, Adoption, or Assessments, ensuring alignment with customer objectives, adhering to ... Show more
As part of Diabsolut’s continued growth, we are expanding our Innovation team with an experienced.It is not a delivery role, a utilization role, or a presales capacity role.It exists so that Diabso... Show more
AVA Energy are partnered with an established firm in the Energy space as they look to hire a Transmission & Interconnection Engineer to bolster their Power Systems offering.This candidate must be a... Show more
This role can be based in US or Canada - EST working hours.Role & Responsibilities Overview:.Architecture & Technical Leadership.Define end-to-end architecture for agentic AI-enabled platform acros... Show more
Elevate your career as a Senior AWS Data Engineer focusing on analytics and BI solutions.Engage in a dynamic hybrid environment to design and deliver data solutions for informed decision-making.You... Show more
As a Data Engineer, you will be working within the Retail Media Platform team to build a best-in-class Ad tech solution that will allow internal and external users to create, manage and report on a... Show more
Navacord is hiring technical talent in the insurance industry! We are accepting applications for a dynamic individual with 5+ years experience in a full stack development environment and with at le... Show more
