Talent.com
Astra North Infoteck Inc.
Principal Data Engineer – DatabricksAstra North Infoteck Inc. • Toronto, Ontario, Canada
Principal Data Engineer – Databricks

Principal Data Engineer – Databricks

Astra North Infoteck Inc. • Toronto, Ontario, Canada
19 days ago
Salary
CA$10.00 hourly
Job type
  • Full-time
Job description

Principal Data Engineer Databricks Spark Delta Lake PySpark Data Lakehouse AWS/Azure

Job Description

Location: Toronto
Work Model: Onsite (4 days/week)

Key Requirements

1218 years of overall Data Engineering experience.
8 years of experience with Enterprise Data Warehouse and Data Lake platforms.
5 years of hands-on experience with Databricks and Apache Spark at scale.
Strong experience modernizing legacy Cloudera platforms (CDH/CDP Hive HBase Impala Spark) to Databricks Lakehouse.
Experience redesigning ingestion transformation and consumption patterns from HDFS-based architecture to cloud object storage and Delta Lake.
Experience refactoring legacy Hive/Impala logic into PySpark and Spark SQL ELT pipelines.
Experience ensuring data reconciliation audit integrity and consistency during migration.
Experience designing and governing Enterprise Data Warehouse and Data Lake/Lakehouse architectures.
Experience implementing layered architectures including:
Raw/Landing Layer
Curated/Conformed Layer
Semantic/Consumption Layer
Experience modernizing traditional Enterprise Data Warehouse platforms into scalable Lakehouse architectures.
Strong experience with finance and risk data models including:
General Ledger
Sub-ledger
Financial Hierarchies
Credit Risk Models
Liquidity Risk Models
Market Risk Models
Experience enabling reporting use cases including aggregation drill-down and drill-back capabilities.
Experience building and managing semantic/consumption layers for BI reporting and analytics.
Ability to define business metrics dimensions hierarchies and KPIs.
Experience with Databricks SQL Delta Tables and dbt or similar frameworks.
Strong experience developing and optimizing large-scale data pipelines using:
PySpark
Spark SQL
Delta Lake
Experience implementing Medallion Architecture:
Bronze Layer
Silver Layer
Gold Layer
Experience optimizing workloads using Z-ORDER OPTIMIZE caching and cluster configurations.
Experience implementing data governance data quality frameworks reconciliation controls and exception handling.
Experience establishing data lineage and metadata management.
Knowledge of data security access control and compliance standards.
Experience with cloud platforms such as AWS or Azure.
Experience with CI/CD pipelines using:
Git
Terraform
Jenkins
Azure DevOps
Familiarity with orchestration tools such as:
Apache Airflow
Databricks Workflows
Experience with dbt is a plus.
Ability to act as a technical authority and lead architecture decisions.
Experience mentoring senior engineers and establishing engineering standards.
Strong stakeholder management skills with finance risk analytics and governance teams.
Ability to translate complex data structures into business-ready insights.

Nice to Have

Experience in Banking Financial Services Insurance (BFSI) Capital Markets or regulatory reporting.
Exposure to:
SAP Finance
Oracle Financials
SAP S/4HANA
Experience supporting AI/ML workloads.
Databricks or cloud certifications.

Key Responsibilities

Lead Cloudera to Databricks transformation initiatives.
Design and implement enterprise Data Lakehouse and Data Warehouse solutions.
Build scalable high-performance data pipelines and modern data architectures.
Drive data modernization governance quality and security initiatives.
Support regulatory management and analytical reporting platforms.
Provide technical leadership mentor engineering teams and establish best practices.




Required Skills:

Sailpoint


Employment Type : Full Time
Experience: years
Vacancy: 1
Monthly Salary Salary: 10 - 10
Create a job alert for this search

Principal Data Engineer – Databricks • Toronto, Ontario, Canada

Similar jobs

Principal Data Engineer (Azure)

Tiger AnalyticsToronto, ON, CA
Full-time

Be among the first 25 applicants.Get AI-powered advice on this job and more exclusive features.Tiger Analytics is a global AI and analytics consulting firm.With data and technology at the core of o... Show more

 • Promoted

Principal Data Engineer

Fitch RatingsToronto
Full-time

Fitch Group is currently seeking a Associate Director/Lead Data Engineer based out of our Toronto office.As a leading, global financial information services provider, Fitch Group delivers vital cre... Show more

 • Promoted

Calabrio Principal Software Engineer – Data & Analytics

TensecToronto, ON, CA
Full-time

Join Calabrio as a Principal Software Engineer focusing on Data and Analytics.This position incorporates innovative BI solutions and data management using AWS technologies.As a Principal Software E... Show more

 • Promoted

Oliver Wyman - Principal Data Engineer - $240,000 - $260,000 A Year

Oliver WymanEast York, Canada
Full-time

The Principal Data Engineer will lead teams, design solutions, and deliver technology strategies for clients, requiring extensive software engineering experience. Show more

 • Promoted

Senior Databricks Data Engineer - Lakehouse - C$120,000 - C$150,000 A Year

Citco GSGSEast York, Canada
Full-time

Seeking a Senior Data Engineer with Databricks, Python, and AWS experience to optimize data pipelines. Show more

 • Promoted

Senior Data Engineer — Aws/Databricks Data Platform Lead - $90,000 - $140,000 A Year

Vanguard CanadaNorth York, Canada
Full-time

Senior Data Engineer role in Toronto.Design and optimize data pipelines using AWS and Databricks.Requires strong SQL and Python skills. Show more

 • Promoted

Affinity Principal Engineer Data Solutions

AffinityToronto
Full-time

Excel in a hands-on leadership role as a Principal Engineer of Data Solutions with Affinity, offering a hybrid work model in Toronto.Drive quality and innovation in analytics engineering.Affinity i... Show more

 • Promoted

Principal Software Engineer - Data

ZoomInfotoronto, on, Canada
Full-time

We are seeking an experienced Principal Software Engineer (Data) to join our fast-paced, collaborative data team.In this role, you will have broad authority to drive the direction of our technograp... Show more

 • Promoted

Principal Cloud Engineer

United States Digital Space LLCtoronto, on, Canada
Full-time

Join a purpose driven winning team, committed to results, in an inclusive and high-performing culture.One of the main areas of transformation is the public cloud and the various platform technologi... Show more

 • Promoted

Principal Data Engineer - C$140,000 - C$180,000 A Year - Remote

RavlNorth York, Canada
Remote
Full-time

RAVL seeks a Principal Data Engineer to design, build, and lead enterprise-grade data and AI platforms.This role focuses on creating scalable, secure platforms and driving innovation across client ... Show more

 • Promoted

Data Platform Engineer

ApexonToronto, Canada
Full-time

Build and scale a modern financial data platform using Python, FastAPI, Snowflake, and cloud-native backend engineering, designing scalable backend services and data pipelines. Show more

 • Promoted

Principal Data Architect - C$184,000 - C$205,000 A Year

PointClickCareNorth York, Canada
Full-time

Lead the definition and execution of a unified data strategy, ensuring scalable, future-proof data architecture for AI and analytics initiatives.Focus on standards, persistence, technology evaluati... Show more

 • Promoted

Principal Data Engineer

AvisoToronto, ON, CA
Full-time

We are looking for a Principal Data Engineer to join our Data and Analytics Group within the Technology Services Partners (TSP) department.Reporting to the Sr Director, Data Platform Delivery and E... Show more

 • Promoted

Databricks Engineer

PeoplePilottoronto, on, Canada
Full-time

Data Warehouse and Data Lake platforms.Strong experience in modernizing legacy Cloudera platforms (CDH/CDP, Hive, HBase, Impala, Spark) to Databricks Lakehouse.Redesign ingestion, transformation, a... Show more

 • Promoted

Principal Engineer, Enterprise Data & Analytics - $120,000 - $150,000 A Year

QuadReal Property GroupToronto County, Canada
Full-time

Senior technical leader responsible for data and analytics solutions, setting standards, mentoring, and hands-on delivery of complex data problems for a global real estate investment company. Show more

 • Promoted

Principal Engineer — Ai-Driven Data Extraction Platform - C$202,000 - C$278,000 A Year

Leading Data Intelligence FirmNorth York, Canada
Full-time

Lead the design and reliability of a global financial data extraction platform, focusing on AI-driven solutions and mentoring engineers. Show more

 • Promoted

Senior Databricks Data Engineer — Lakehouse & Ai Pipelines - C$120,000 - C$150,000 A Year

Financial Services ProviderEast York, Canada
Full-time

Data Engineer needed in Toronto to design and optimize data pipelines on Databricks Lakehouse Platform using Python, Spark, and AWS for a financial services provider. Show more

 • Promoted

Principal Data Engineer - Tangerine

Scotiabanktoronto, on, Canada
Full-time

Tangerine is Canada’s leading direct bank.We offer flexible and accessible banking options, innovative products, and award-winning client service.Our employees come to work each day to help Canadia... Show more

 • Promoted

Principal Engineer — Ai-Driven Data Extraction Platform - C$202,000 - C$278,000 A Year

Data Intelligence FirmNorth York, Canada
Full-time

Lead the design and reliability of a global financial data extraction platform, focusing on AI-driven solutions and system architecture for a leading data intelligence firm. Show more

 • Promoted

Principal Data Solutions Engineer - C$120,000 - C$150,000 A Year

Leading Real Estate FirmNorth York, Canada
Full-time

Seeking a Principal Data Solutions Engineer for a leading real estate firm to lead data analytics solutions, provide mentorship, and develop technical standards using SQL and Python. Show more