Talent.com
Astra North Infoteck Inc.
Principal Databricks Data EngineerAstra North Infoteck Inc. • Toronto, ON, ca
Principal Databricks Data Engineer

Principal Databricks Data Engineer

Astra North Infoteck Inc. • Toronto, ON, ca
5 days ago
Job type
  • Full-time
  • Quick Apply
Job description
Job Description

Principal Databricks Data Engineer

Experience Required: 12–18 Years

Key Requirements

• 12–18 years of overall Data Engineering experience
• 8+ years of experience with enterprise Data Warehouse and Data Lake platforms
• 5+ years of hands-on experience with Databricks and Apache Spark at scale
• Strong experience modernizing legacy Cloudera platforms including:
• CDH/CDP
• Hive
• HBase
• Impala
• Spark
• Modernize Cloudera platforms to Databricks Lakehouse architecture
• Redesign ingestion, transformation, and consumption patterns from HDFS-based architecture to Cloud Object Storage and Delta Lake
• Refactor legacy Hive and Impala logic into PySpark and Spark SQL ELT pipelines
• Ensure data reconciliation, audit integrity, and consistency during migration
• Design and govern enterprise Data Warehouse and Data Lake/Lakehouse architectures
• Implement layered data architecture including:
• Raw / Landing Layer
• Curated / Conformed Layer
• Semantic / Consumption Layer
• Modernize traditional Enterprise Data Warehouse platforms into scalable Lakehouse architectures
• Strong experience with finance and risk data models including:
• General Ledger
• Sub-ledger
• Financial Hierarchies
• Credit Risk
• Liquidity Risk
• Market Risk
• Enable reporting capabilities including:
• Aggregation
• Drill-down
• Drill-back
• Build and manage semantic and consumption layers for BI, reporting, and analytics
• Define business metrics, dimensions, hierarchies, and KPIs
• Experience with:
• Databricks SQL
• Delta Tables
• dbt or similar frameworks
• Develop and optimize large-scale data pipelines using:
• PySpark
• Spark SQL
• Delta Lake
• Implement Medallion Architecture including:
• Bronze Layer
• Silver Layer
• Gold Layer
• Optimize workloads using:
• Z-ORDER
• OPTIMIZE
• Caching
• Cluster Configuration Tuning
• Implement:
• Data Governance
• Data Quality Frameworks
• Reconciliation Controls
• Exception Handling
• Establish data lineage and metadata management
• Ensure data security, access control, and compliance standards
• Experience with AWS or Azure cloud platforms
• Experience with CI/CD pipelines using:
• Git
• Terraform
• Jenkins
• Azure DevOps
• Familiarity with:
• Apache Airflow
• Databricks Workflows
• Experience with dbt is an advantage
• Act as a technical authority and lead enterprise architecture decisions
• Mentor senior engineers and establish engineering standards
• Collaborate with finance, risk, analytics, and governance stakeholders
• Translate complex data structures into business-ready insights

Nice-to-Have Skills

• Experience in BFSI, Capital Markets, or Regulatory Reporting
• Exposure to:
• SAP Finance
• Oracle Financials
• SAP S/4HANA
• Experience supporting AI/ML workloads
• Databricks and Cloud Certifications

Key Responsibilities

• Lead Cloudera to Databricks transformation initiatives
• Shape enterprise finance and risk data platforms
• Support regulatory, management, and analytical reporting systems

Essential Skills

• Databricks
• Apache Spark
• PySpark
• Spark SQL
• Delta Lake
• Lakehouse Architecture
• Cloudera (CDH/CDP)
• Hive
• HBase
• Impala
• Data Warehouse & Data Lake
• Medallion Architecture
• Databricks SQL
• dbt
• AWS / Azure
• Airflow / Databricks Workflows
• CI/CD
• Terraform
• Jenkins
• Git
• Finance & Risk Data Models
• Enterprise Data Architecture








Requirements
Sailpoint
Create a job alert for this search

Principal Databricks Data Engineer • Toronto, ON, ca

Similar jobs

Principal Data Engineer (Azure)

Tiger AnalyticsToronto, ON, CA
Full-time

Be among the first 25 applicants.Get AI-powered advice on this job and more exclusive features.Tiger Analytics is a global AI and analytics consulting firm.With data and technology at the core of o... Show more

 • Promoted

Principal Software Engineer - Data

ZoomInfoToronto
Full-time

We are seeking an experienced Principal Software Engineer (Data) to join our fast-paced, collaborative data team.In this role, you will have broad authority to drive the direction of our technograp... Show more

 • Promoted

Big Data Engineer

TMX GroupToronto, ON, CA
Full-time

We are seeking a seasoned and motivated Big Data Engineer to join our growing Enterprise Data and Advanced Analytics Platform team.The ideal candidate is an expert data pipeline builder and data wr... Show more

 • Promoted

Principal Data Engineer

RAVLToronto
Full-time

RAVL helps technologists accelerate their careers.At RAVL, we connect strategy with execution, care deeply about the people we work with, and measure success by the lasting impact we leave behind.O... Show more

 • Promoted

Principal Software Engineer, BI & Data Platform

CalabrioToronto, ON, CA
Full-time

Are you driven by innovation and looking to thrive in a fast-paced, growing environment? Join us at Calabrio‑Verint and be part of our dynamic team! Help us in reshaping the landscape of customer e... Show more

 • Promoted

Principal Data Engineer

AvisoToronto, Ontario, Canada
Full-time

We are looking for a Principal Data Engineer to join our Data and Analytics Group within the Technology Services Partners (TSP) department.Reporting to the Sr Director, Data Platform Delivery and E... Show more

 • Promoted

Senior Data Engineer - Databricks & Azure for Banking

Soar ConsultantsToronto, ON, CA
Full-time

A leading data consulting firm is seeking a Data Engineer for a client in Toronto.The role requires experience with Azure, Databricks, and strong skills in data analysis and integration methodologi... Show more

 • Promoted

Oliver Wyman - Principal Data Engineer - $240,000 - $260,000 A Year

Oliver WymanToronto, Canada
Full-time

Join to apply for theOliver Wyman - Principal Data Engineerrole atOliver WymanLocations: Boston, New York, Chicago, Washington DC, Toronto, MontrealWho We AreOliver Wyman Performance Transformation... Show more

 • Promoted

Senior Data Engineer (Databricks Focused)

BDO CanadaToronto, ON, CA
Full-time

Putting people first, every day.BDO is a firm built on a foundation of positive relationships with our people and our clients.Each day, our professionals provide exceptional service, helping client... Show more

 • Promoted

Junior Databricks Data Engineer — Hybrid Lakehouse - C$70,000 - C$94,000 A Year

The Citco GroupToronto, Canada
Full-time

A leading financial services firm in Toronto is seeking a Junior Data Engineer to contribute to their Centralized Data Platform built on Databricks.The candidate will develop and maintain data pipe... Show more

 • Promoted

Data Engineer

LeadingtalentMarkham, ON, CA
Full-time

At IBM, work is more than a job – it’s a calling: To build.To think along with clients and sell.Not just to do something better, but to attempt things you’ve never thought possible.Are you ready to... Show more

 • Promoted

Senior Data Engineer ( Databricks/Snowflake)

TEEMA Solutions GroupToronto, ON, CA
Full-time

Toronto (3 days downtown/Hybrid).Our client is interviewing to onboard a Senior Data Engineers to join their Data & Analytics team within the Wealth Technology group.We are looking for Data Enginee... Show more

 • Promoted

Principal Data Architect for Advanced Analytics

AdastraToronto, ON, CA
Full-time

Drive data innovation at Adastra as a Principal Data Architect, working in a hybrid model.Lead the evolution of cutting-edge data platforms using AWS and Knowledge Graph technology.In this senior r... Show more

 • Promoted

Data Engineer: PySpark, Databricks & Tableau

Highbrow LLCToronto, ON, CA
Temporary

A leading data engineering firm in Toronto is seeking a Data Engineer with 2-5 years of experience in technologies such as PySpark, SQL, and Databricks.The role involves working in the Data Science... Show more

 • Promoted

Senior Databricks Data Engineer — Lakehouse & Ai Pipelines - C$120,000 - C$150,000 A Year

The Citco Group LimitedToronto, Canada
Full-time

Senior Data Engineer sought to design and optimize data pipelines on Databricks Lakehouse Platform. Show more

 • Promoted

Lead Data Engineer – Databricks

TEEMAToronto, Ontario, Canada
Full-time

Team Lead, Data Engineering – Databricks.Our client has provided specialist financial services to alternative investment funds, investors, multinationals, and private clients worldwide.Our data eng... Show more

 • Promoted

Senior AWS Databricks Data Engineer - Hybrid Toronto

AkkodisToronto, ON, CA
Full-time

A leading IT consultancy is seeking a Mid-Senior Level Data Engineer for a contract role in Toronto.Candidates will design, develop, and maintain scalable data pipelines, ensuring data quality and ... Show more

 • Promoted

Senior Data Engineer – Databricks & Cloud Data Platforms

ShyftLabsToronto
Full-time

A leading data solutions provider in Toronto is seeking a skilled Data Engineer to design and optimize big data solutions using Databricks.The ideal candidate will have significant experience with ... Show more

 • Promoted

Advanced Data Engineer for Large Datasets and ETL Pipelines

LumenaltaToronto, ON, CA
Full-time

Transform data into insights as an Advanced Data Engineer specializing in complex ETL pipelines.Embrace this remote opportunity to use your Python or Java and SQL skills effectively.This role invit... Show more

 • Promoted

Lead Data Engineer

Thomson ReutersToronto, ON, CA
Full-time

We are seeking a highly skilled and strategic Lead Data Engineer to join our team.In this advanced individual contributor role, you will spearhead the development of sophisticated, insightful, and ... Show more