Databricks Developer
- Temps plein
- Quick Apply
Job Title: Senior Databricks Developer
Location: Toronto, ON
Work Mode: Hybrid
Experience: 8 10 Years
Primary Skills: Databricks, PySpark/Spark SQL, Delta Lake
Job Description
We are seeking an experienced Databricks Developer with strong expertise in building, optimizing, and governing scalable data pipelines on the Databricks Lakehouse platform.
The ideal candidate should have hands-on experience with PySpark, Spark SQL, Delta Lake, Lakehouse architecture, data orchestration, governance, and CI/CD automation.
Key Responsibilities
-
ETL Pipeline Development: Design and build scalable batch and streaming data pipelines using PySpark, Spark SQL, and Delta Lake.
-
Lakehouse Architecture: Implement medallion architectures (Bronze, Silver, Gold layers) and Delta Live Tables (DLT).
-
Orchestration & Automation: Build and schedule production Databricks jobs, workflows, and pipelines using tools such as Azure Data Factory (ADF) or Databricks Workflows.
-
Governance & Security: Configure and manage access control, metadata, and data governance using Unity Catalog and CI/CD practices.
-
DevOps: Maintain source control using Git and automated deployment pipelines using Azure DevOps, GitHub Actions, or Databricks Asset Bundles.
-
Build, tune, optimize, and govern data pipelines on the Databricks Lakehouse platform.
-
Ensure data quality, pipeline reliability, performance, and scalability.
-
Collaborate with data engineering and business teams to deliver enterprise data solutions.
Required Skills
-
Strong hands-on experience with Databricks Lakehouse Platform.
-
Expertise in PySpark and Spark SQL.
-
Experience with Delta Lake and Delta Live Tables (DLT).
-
Knowledge of Medallion Architecture (Bronze, Silver, Gold).
-
Experience with batch and streaming ETL pipelines.
-
Hands-on experience with Azure Data Factory or Databricks Workflows.
-
Knowledge of Unity Catalog and data governance.
-
Experience with Git, Azure DevOps, GitHub Actions, and Databricks Asset Bundles.
-
Strong understanding of CI/CD and data pipeline optimization.
Preferred Experience
-
8 10 years of overall experience.
-
Enterprise-scale data engineering and Lakehouse implementation experience.
-
Experience developing and maintaining production-grade Databricks pipelines.