Talent.com
Astra North Infoteck Inc.
Principal Databricks Data EngineerAstra North Infoteck Inc. • Toronto, Ontario, Canada
Principal Databricks Data Engineer

Principal Databricks Data Engineer

Astra North Infoteck Inc. • Toronto, Ontario, Canada
24 days ago
Salary
CA$10.00 hourly
Job type
  • Full-time
Job description

Principal Databricks Data Engineer

Experience Required: 1218 Years

Key Requirements

1218 years of overall Data Engineering experience
8 years of experience with enterprise Data Warehouse and Data Lake platforms
5 years of hands-on experience with Databricks and Apache Spark at scale
Strong experience modernizing legacy Cloudera platforms including:
CDH/CDP
Hive
HBase
Impala
Spark
Modernize Cloudera platforms to Databricks Lakehouse architecture
Redesign ingestion transformation and consumption patterns from HDFS-based architecture to Cloud Object Storage and Delta Lake
Refactor legacy Hive and Impala logic into PySpark and Spark SQL ELT pipelines
Ensure data reconciliation audit integrity and consistency during migration
Design and govern enterprise Data Warehouse and Data Lake/Lakehouse architectures
Implement layered data architecture including:
Raw / Landing Layer
Curated / Conformed Layer
Semantic / Consumption Layer
Modernize traditional Enterprise Data Warehouse platforms into scalable Lakehouse architectures
Strong experience with finance and risk data models including:
General Ledger
Sub-ledger
Financial Hierarchies
Credit Risk
Liquidity Risk
Market Risk
Enable reporting capabilities including:
Aggregation
Drill-down
Drill-back
Build and manage semantic and consumption layers for BI reporting and analytics
Define business metrics dimensions hierarchies and KPIs
Experience with:
Databricks SQL
Delta Tables
dbt or similar frameworks
Develop and optimize large-scale data pipelines using:
PySpark
Spark SQL
Delta Lake
Implement Medallion Architecture including:
Bronze Layer
Silver Layer
Gold Layer
Optimize workloads using:
Z-ORDER
OPTIMIZE
Caching
Cluster Configuration Tuning
Implement:
Data Governance
Data Quality Frameworks
Reconciliation Controls
Exception Handling
Establish data lineage and metadata management
Ensure data security access control and compliance standards
Experience with AWS or Azure cloud platforms
Experience with CI/CD pipelines using:
Git
Terraform
Jenkins
Azure DevOps
Familiarity with:
Apache Airflow
Databricks Workflows
Experience with dbt is an advantage
Act as a technical authority and lead enterprise architecture decisions
Mentor senior engineers and establish engineering standards
Collaborate with finance risk analytics and governance stakeholders
Translate complex data structures into business-ready insights

Nice-to-Have Skills

Experience in BFSI Capital Markets or Regulatory Reporting
Exposure to:
SAP Finance
Oracle Financials
SAP S/4HANA
Experience supporting AI/ML workloads
Databricks and Cloud Certifications

Key Responsibilities

Lead Cloudera to Databricks transformation initiatives
Shape enterprise finance and risk data platforms
Support regulatory management and analytical reporting systems

Essential Skills

Databricks
Apache Spark
PySpark
Spark SQL
Delta Lake
Lakehouse Architecture
Cloudera (CDH/CDP)
Hive
HBase
Impala
Data Warehouse & Data Lake
Medallion Architecture
Databricks SQL
dbt
AWS / Azure
Airflow / Databricks Workflows
CI/CD
Terraform
Jenkins
Git
Finance & Risk Data Models
Enterprise Data Architecture








Required Skills:

Sailpoint


Employment Type : Full Time
Experience: years
Vacancy: 1
Monthly Salary Salary: 10 - 10
Create a job alert for this search

Principal Databricks Data Engineer • Toronto, Ontario, Canada

Similar jobs

Affinity Principal Engineer Data Solutions

AffinityToronto, ON, CA
Full-time

Excel in a hands-on leadership role as a Principal Engineer of Data Solutions with Affinity, offering a hybrid work model in Toronto.Drive quality and innovation in analytics engineering.Affinity i... Show more

 • Promoted

Principal Data Engineer (Azure)

Tiger AnalyticsToronto, ON, CA
Full-time

Be among the first 25 applicants.Get AI-powered advice on this job and more exclusive features.Tiger Analytics is a global AI and analytics consulting firm.With data and technology at the core of o... Show more

 • Promoted

Databricks Platform Engineer

BrokerLinkToronto
Full-time

Multiple benefits offered to support physical and mental wellbeing, Wellness account and much more.Salary Range (but Not Limited To): 101,800 - 124,400.Incentive plan: Annual bonus: 12% target, bas... Show more

 • Promoted

Data Engineer: PySpark, Databricks & Tableau

Highbrow LLCToronto
Full-time +1

A leading data engineering firm in Toronto is seeking a Data Engineer with 2-5 years of experience in technologies such as PySpark, SQL, and Databricks.The role involves working in the Data Science... Show more

 • Promoted

Oliver Wyman - Principal Data Engineer - $240,000 - $260,000 A Year

Oliver WymanEast York, Canada
Full-time

The Principal Data Engineer will lead teams, design solutions, and deliver technology strategies for clients, requiring extensive software engineering experience. Show more

 • Promoted

Principal Engineer — Ai-Driven Data Extraction Platform - C$202,000 - C$278,000 A Year

Leading Data Intelligence FirmEast York, Canada
Full-time

Lead the design and reliability of a global financial data extraction platform, focusing on AI-driven solutions and mentoring engineers. Show more

 • Promoted

Junior Databricks Data Engineer: Aws Lakehouse Pipelines - C$70,000 - C$94,000 A Year

Specialist Financial Services FirmNorth York, Canada
Full-time

Junior Data Engineer sought by a financial services firm in Toronto to build and optimize data pipelines on Databricks and AWS, requiring 1+ year experience with Python, Spark, and AWS. Show more

 • Promoted

Principal Data Engineer

Fitch RatingsToronto, ON, CA
Full-time

Fitch Group is currently seeking a Associate Director/Lead Data Engineer based out of our Toronto office.As a leading, global financial information services provider, Fitch Group delivers vital cre... Show more

 • Promoted

Principal Software Engineer - Data

ZoomInfotoronto, on, Canada
Full-time

We are seeking an experienced Principal Software Engineer (Data) to join our fast-paced, collaborative data team.In this role, you will have broad authority to drive the direction of our technograp... Show more

 • Promoted

Databricks Engineer

PeoplePilottoronto, on, Canada
Full-time

Data Warehouse and Data Lake platforms.Strong experience in modernizing legacy Cloudera platforms (CDH/CDP, Hive, HBase, Impala, Spark) to Databricks Lakehouse.Redesign ingestion, transformation, a... Show more

 • Promoted

Principal Data Engineer

AvisoToronto, ON, CA
Full-time

We are looking for a Principal Data Engineer to join our Data and Analytics Group within the Technology Services Partners (TSP) department.Reporting to the Sr Director, Data Platform Delivery and E... Show more

 • Promoted

Junior Databricks Data Engineer — Hybrid Lakehouse - C$70,000 - C$94,000 A Year

Leading Financial Services FirmNorth York, Canada
Full-time

Junior Data Engineer wanted for a financial firm in Toronto to build and manage data platforms, focusing on ETL/ELT optimization and data governance in a hybrid setting. Show more

 • Promoted

Principal Engineer, Enterprise Data & Analytics - $120,000 - $150,000 A Year

QuadReal Property GroupNorth York, Canada
Full-time

Senior technical leader responsible for data and analytics solutions, setting standards, mentoring, and hands-on delivery of complex data problems for a global real estate investment company. Show more

 • Promoted

Principal Engineer Data Solutions

AffinityToronto, Canada
Full-time

Location: Preference to have candidates based in Toronto, next best, candidates would be located in Vancouver.Hybrid work environment of 1 - 2 days onsite.On behalf of our client, Affinity is seeki... Show more

 • Promoted

Senior Databricks Data Engineer — Lakehouse & Ai Pipelines - C$120,000 - C$150,000 A Year

Financial Services ProviderEast York, Canada
Full-time

Data Engineer needed in Toronto to design and optimize data pipelines on Databricks Lakehouse Platform using Python, Spark, and AWS for a financial services provider. Show more

 • Promoted

Senior Databricks Data Engineer - Lakehouse - C$120,000 - C$150,000 A Year

Citco GSGSNorth York, Canada
Full-time

Seeking a Senior Data Engineer with Databricks, Python, and AWS experience to optimize data pipelines. Show more

 • Promoted

Principal Data Engineer - Tangerine

Scotiabanktoronto, on, Canada
Full-time

Tangerine is Canada’s leading direct bank.We offer flexible and accessible banking options, innovative products, and award-winning client service.Our employees come to work each day to help Canadia... Show more

 • Promoted

Principal Engineer — Ai-Driven Data Extraction Platform - C$202,000 - C$278,000 A Year

Data Intelligence FirmNorth York, Canada
Full-time

Lead the design and reliability of a global financial data extraction platform, focusing on AI-driven solutions and system architecture for a leading data intelligence firm. Show more

 • Promoted

Principal Data Solutions Engineer - C$120,000 - C$150,000 A Year

Leading Real Estate FirmNorth York, Canada
Full-time

Seeking a Principal Data Solutions Engineer for a leading real estate firm to lead data analytics solutions, provide mentorship, and develop technical standards using SQL and Python. Show more

 • Promoted

Senior Databricks Data Engineer — Lakehouse & Ai Pipelines - C$120,000 - C$150,000 A Year

A leading financial services providerToronto County, Canada
Full-time

Senior Data Engineer needed in Toronto to build and optimize data pipelines on Databricks, using Python, Spark, and AWS for financial data insights. Show more