Azure Databricks Platform engineer
- Temps plein
- Quick Apply
Job Title: Azure Databricks Platform Engineer
Location: Toronto, ON
Work Mode: Hybrid
Job Summary:
We are seeking an experienced Azure Databricks Platform Engineer to design, implement, administer, automate, and support enterprise-scale Azure Databricks environments. The ideal candidate will have strong expertise in cloud infrastructure, platform engineering, security, governance, and DevOps automation.
Key Responsibilities:
-
Design, configure, administer, and support Azure Databricks workspaces across development, testing, UAT, and production environments.
-
Manage Databricks compute, cluster policies, jobs, workflows, user access, monitoring, and cost optimization.
-
Implement and administer Unity Catalog, including catalogs, schemas, permissions, storage credentials, and external locations.
-
Integrate Azure Databricks with ADLS Gen2, Azure Data Factory, Azure SQL, Azure Key Vault, and managed identities.
-
Implement secure authentication and authorization using Microsoft Entra ID, Azure RBAC, and managed identities.
-
Manage Azure networking, including VNets, NSGs, private endpoints, DNS, routing, VPN, ExpressRoute, and hub-and-spoke connectivity.
-
Automate platform operations using Python, PowerShell, Bash, Terraform, and Bicep.
-
Support CI/CD and release management using Azure DevOps, GitHub Actions, or equivalent tools.
-
Establish platform monitoring and observability using Azure Monitor, Log Analytics, Grafana, Prometheus, and other monitoring tools.
-
Troubleshoot complex Databricks, infrastructure, storage, identity, networking, and connectivity issues.
-
Implement platform security, governance, access controls, and operational standards.
Required Skills:
-
Strong experience administering and supporting enterprise Azure Databricks environments.
-
Expertise in Unity Catalog, Databricks compute, cluster policies, jobs, workflows, and platform governance.
-
Experience integrating Databricks with ADLS Gen2, Azure Data Factory, Azure SQL, and Azure Key Vault.
-
Strong understanding of Microsoft Entra ID, Azure RBAC, managed identities, and secure authentication.
-
Experience with Azure networking, private endpoints, VNets, NSGs, DNS, VPN, and ExpressRoute.
-
Proficiency in Python, PowerShell, Bash, Terraform, and Bicep.
-
Experience with Azure DevOps, GitHub Actions, CI/CD, and infrastructure automation.
-
Knowledge of monitoring, observability, troubleshooting, performance optimization, and cost management.
Pre-Screening Questions:
-
Describe your experience designing and administering enterprise Azure Databricks platforms.
-
What is your hands-on experience with Unity Catalog, cluster policies, and workspace administration?
-
How have you integrated Azure Databricks with ADLS Gen2, Azure Data Factory, and Azure Key Vault?
-
Describe your experience implementing secure authentication using Microsoft Entra ID and managed identities.
-
What is your experience with Terraform, Bicep, Python, and CI/CD automation?
-
How have you implemented platform monitoring, observability, and cost optimization?
-
Describe a complex Databricks or Azure infrastructure issue you diagnosed and resolved.