Talent.com
Astra North Infoteck Inc.
Principal Data Engineer – DatabricksAstra North Infoteck Inc. • Toronto, ON, ca
Principal Data Engineer – Databricks

Principal Data Engineer – Databricks

Astra North Infoteck Inc. • Toronto, ON, ca
20 days ago
Job type
  • Full-time
  • Quick Apply
Job description
Job Description

Principal Data Engineer – Databricks | Spark | Delta Lake | PySpark | Data Lakehouse | AWS/Azure

Job Description


Location: Toronto
Work Model: Onsite (4 days/week)


Key Requirements


• 12–18 years of overall Data Engineering experience.
• 8+ years of experience with Enterprise Data Warehouse and Data Lake platforms.
• 5+ years of hands-on experience with Databricks and Apache Spark at scale.
• Strong experience modernizing legacy Cloudera platforms (CDH/CDP, Hive, HBase, Impala, Spark) to Databricks Lakehouse.
• Experience redesigning ingestion, transformation, and consumption patterns from HDFS-based architecture to cloud object storage and Delta Lake.
• Experience refactoring legacy Hive/Impala logic into PySpark and Spark SQL ELT pipelines.
• Experience ensuring data reconciliation, audit integrity, and consistency during migration.
• Experience designing and governing Enterprise Data Warehouse and Data Lake/Lakehouse architectures.
• Experience implementing layered architectures including:
• Raw/Landing Layer
• Curated/Conformed Layer
• Semantic/Consumption Layer
• Experience modernizing traditional Enterprise Data Warehouse platforms into scalable Lakehouse architectures.
• Strong experience with finance and risk data models, including:
• General Ledger
• Sub-ledger
• Financial Hierarchies
• Credit Risk Models
• Liquidity Risk Models
• Market Risk Models
• Experience enabling reporting use cases including aggregation, drill-down, and drill-back capabilities.
• Experience building and managing semantic/consumption layers for BI, reporting, and analytics.
• Ability to define business metrics, dimensions, hierarchies, and KPIs.
• Experience with Databricks SQL, Delta Tables, and dbt or similar frameworks.
• Strong experience developing and optimizing large-scale data pipelines using:
• PySpark
• Spark SQL
• Delta Lake
• Experience implementing Medallion Architecture:
• Bronze Layer
• Silver Layer
• Gold Layer
• Experience optimizing workloads using Z-ORDER, OPTIMIZE, caching, and cluster configurations.
• Experience implementing data governance, data quality frameworks, reconciliation controls, and exception handling.
• Experience establishing data lineage and metadata management.
• Knowledge of data security, access control, and compliance standards.
• Experience with cloud platforms such as AWS or Azure.
• Experience with CI/CD pipelines using:
• Git
• Terraform
• Jenkins
• Azure DevOps
• Familiarity with orchestration tools such as:
• Apache Airflow
• Databricks Workflows
• Experience with dbt is a plus.
• Ability to act as a technical authority and lead architecture decisions.
• Experience mentoring senior engineers and establishing engineering standards.
• Strong stakeholder management skills with finance, risk, analytics, and governance teams.
• Ability to translate complex data structures into business-ready insights.


Nice to Have


• Experience in Banking, Financial Services, Insurance (BFSI), Capital Markets, or regulatory reporting.
• Exposure to:
• SAP Finance
• Oracle Financials
• SAP S/4HANA
• Experience supporting AI/ML workloads.
• Databricks or cloud certifications.


Key Responsibilities


• Lead Cloudera to Databricks transformation initiatives.
• Design and implement enterprise Data Lakehouse and Data Warehouse solutions.
• Build scalable, high-performance data pipelines and modern data architectures.
• Drive data modernization, governance, quality, and security initiatives.
• Support regulatory, management, and analytical reporting platforms.
• Provide technical leadership, mentor engineering teams, and establish best practices.





Requirements
Sailpoint
Create a job alert for this search

Principal Data Engineer – Databricks • Toronto, ON, ca

Similar jobs

Principal Data Engineer (Azure)

Tiger AnalyticsToronto, ON, CA
Full-time

Be among the first 25 applicants.Get AI-powered advice on this job and more exclusive features.Tiger Analytics is a global AI and analytics consulting firm.With data and technology at the core of o... Show more

 • Promoted

Principal Data Engineer

Fitch RatingsToronto
Full-time

Fitch Group is currently seeking a Associate Director/Lead Data Engineer based out of our Toronto office.As a leading, global financial information services provider, Fitch Group delivers vital cre... Show more

 • Promoted

Calabrio Principal Software Engineer – Data & Analytics

TensecToronto, ON, CA
Full-time

Join Calabrio as a Principal Software Engineer focusing on Data and Analytics.This position incorporates innovative BI solutions and data management using AWS technologies.As a Principal Software E... Show more

 • Promoted

Principal Engineer — Ai-Driven Data Extraction Platform - C$202,000 - C$278,000 A Year

Data Intelligence FirmToronto, Canada
Full-time

Lead the design and reliability of a global financial data extraction platform, focusing on AI-driven solutions and system architecture for a leading data intelligence firm. Show more

 • Promoted

Oliver Wyman - Principal Data Engineer - $240,000 - $260,000 A Year

Oliver WymanEast York, Canada
Full-time

The Principal Data Engineer will lead teams, design solutions, and deliver technology strategies for clients, requiring extensive software engineering experience. Show more

 • Promoted

Affinity Principal Engineer Data Solutions

AffinityToronto
Full-time

Excel in a hands-on leadership role as a Principal Engineer of Data Solutions with Affinity, offering a hybrid work model in Toronto.Drive quality and innovation in analytics engineering.Affinity i... Show more

 • Promoted

Principal Data Engineer - C$140,000 - C$180,000 A Year - Remote

RavlEast York, Canada
Remote
Full-time

RAVL seeks a Principal Data Engineer to design, build, and lead enterprise-grade data and AI platforms.This role focuses on creating scalable, secure platforms and driving innovation across client ... Show more

 • Promoted

Senior Databricks Data Engineer — Lakehouse & Ai Pipelines - C$120,000 - C$150,000 A Year

Financial Services ProviderNorth York, Canada
Full-time

Data Engineer needed in Toronto to design and optimize data pipelines on Databricks Lakehouse Platform using Python, Spark, and AWS for a financial services provider. Show more

 • Promoted

Principal Data Architect - C$184,000 - C$205,000 A Year

PointClickCareToronto, Canada
Full-time

Lead the definition and execution of a unified data strategy, ensuring scalable, future-proof data architecture for AI and analytics initiatives.Focus on standards, persistence, technology evaluati... Show more

 • Promoted

Senior Data Engineer — Aws/Databricks Data Platform Lead - $90,000 - $140,000 A Year

Vanguard CanadaEast York, Canada
Full-time

Senior Data Engineer role in Toronto.Design and optimize data pipelines using AWS and Databricks.Requires strong SQL and Python skills. Show more

 • Promoted

Principal Software Engineer - Data

ZoomInfotoronto, on, Canada
Full-time

We are seeking an experienced Principal Software Engineer (Data) to join our fast-paced, collaborative data team.In this role, you will have broad authority to drive the direction of our technograp... Show more

 • Promoted

Principal Cloud Engineer

United States Digital Space LLCtoronto, on, Canada
Full-time

Join a purpose driven winning team, committed to results, in an inclusive and high-performing culture.One of the main areas of transformation is the public cloud and the various platform technologi... Show more

 • Promoted

Data Platform Engineer

ApexonToronto, Canada
Full-time

Build and scale a modern financial data platform using Python, FastAPI, Snowflake, and cloud-native backend engineering, designing scalable backend services and data pipelines. Show more

 • Promoted

Principal Data Engineer

AvisoToronto, ON, CA
Full-time

We are looking for a Principal Data Engineer to join our Data and Analytics Group within the Technology Services Partners (TSP) department.Reporting to the Sr Director, Data Platform Delivery and E... Show more

 • Promoted

Databricks Engineer

PeoplePilottoronto, on, Canada
Full-time

Data Warehouse and Data Lake platforms.Strong experience in modernizing legacy Cloudera platforms (CDH/CDP, Hive, HBase, Impala, Spark) to Databricks Lakehouse.Redesign ingestion, transformation, a... Show more

 • Promoted

Principal Engineer, Enterprise Data & Analytics - $120,000 - $150,000 A Year

QuadReal Property GroupNorth York, Canada
Full-time

Senior technical leader responsible for data and analytics solutions, setting standards, mentoring, and hands-on delivery of complex data problems for a global real estate investment company. Show more

 • Promoted

Principal Engineer — Ai-Driven Data Extraction Platform - C$202,000 - C$278,000 A Year

Leading Data Intelligence FirmNorth York, Canada
Full-time

Lead the design and reliability of a global financial data extraction platform, focusing on AI-driven solutions and mentoring engineers. Show more

 • Promoted

Senior Databricks Data Engineer - Lakehouse - C$120,000 - C$150,000 A Year

Citco GSGSNorth York, Canada
Full-time

Seeking a Senior Data Engineer with Databricks, Python, and AWS experience to optimize data pipelines. Show more

 • Promoted

Principal Data Engineer - Tangerine

Scotiabanktoronto, on, Canada
Full-time

Tangerine is Canada’s leading direct bank.We offer flexible and accessible banking options, innovative products, and award-winning client service.Our employees come to work each day to help Canadia... Show more

 • Promoted

Principal Data Solutions Engineer - C$120,000 - C$150,000 A Year

Leading Real Estate FirmNorth York, Canada
Full-time

Seeking a Principal Data Solutions Engineer for a leading real estate firm to lead data analytics solutions, provide mentorship, and develop technical standards using SQL and Python. Show more