Talent.com
Acestack
Azure Databricks Platform engineerAcestack • Toronto, ON, Canada
Search for other jobs
Azure Databricks Platform engineer

Azure Databricks Platform engineer

Acestack • Toronto, ON, Canada
6 hours ago
Job type
  • Full-time
  • Quick Apply
Job description

Job Title: Azure Databricks Platform Engineer

Location: Toronto, ON

Work Mode: Hybrid

Job Summary:

We are seeking an experienced Azure Databricks Platform Engineer to design, implement, administer, automate, and support enterprise-scale Azure Databricks environments. The ideal candidate will have strong expertise in cloud infrastructure, platform engineering, security, governance, and DevOps automation.

Key Responsibilities:

  • Design, configure, administer, and support Azure Databricks workspaces across development, testing, UAT, and production environments.

  • Manage Databricks compute, cluster policies, jobs, workflows, user access, monitoring, and cost optimization.

  • Implement and administer Unity Catalog, including catalogs, schemas, permissions, storage credentials, and external locations.

  • Integrate Azure Databricks with ADLS Gen2, Azure Data Factory, Azure SQL, Azure Key Vault, and managed identities.

  • Implement secure authentication and authorization using Microsoft Entra ID, Azure RBAC, and managed identities.

  • Manage Azure networking, including VNets, NSGs, private endpoints, DNS, routing, VPN, ExpressRoute, and hub-and-spoke connectivity.

  • Automate platform operations using Python, PowerShell, Bash, Terraform, and Bicep.

  • Support CI/CD and release management using Azure DevOps, GitHub Actions, or equivalent tools.

  • Establish platform monitoring and observability using Azure Monitor, Log Analytics, Grafana, Prometheus, and other monitoring tools.

  • Troubleshoot complex Databricks, infrastructure, storage, identity, networking, and connectivity issues.

  • Implement platform security, governance, access controls, and operational standards.

Required Skills:

  • Strong experience administering and supporting enterprise Azure Databricks environments.

  • Expertise in Unity Catalog, Databricks compute, cluster policies, jobs, workflows, and platform governance.

  • Experience integrating Databricks with ADLS Gen2, Azure Data Factory, Azure SQL, and Azure Key Vault.

  • Strong understanding of Microsoft Entra ID, Azure RBAC, managed identities, and secure authentication.

  • Experience with Azure networking, private endpoints, VNets, NSGs, DNS, VPN, and ExpressRoute.

  • Proficiency in Python, PowerShell, Bash, Terraform, and Bicep.

  • Experience with Azure DevOps, GitHub Actions, CI/CD, and infrastructure automation.

  • Knowledge of monitoring, observability, troubleshooting, performance optimization, and cost management.

Pre-Screening Questions:

  1. Describe your experience designing and administering enterprise Azure Databricks platforms.

  2. What is your hands-on experience with Unity Catalog, cluster policies, and workspace administration?

  3. How have you integrated Azure Databricks with ADLS Gen2, Azure Data Factory, and Azure Key Vault?

  4. Describe your experience implementing secure authentication using Microsoft Entra ID and managed identities.

  5. What is your experience with Terraform, Bicep, Python, and CI/CD automation?

  6. How have you implemented platform monitoring, observability, and cost optimization?

  7. Describe a complex Databricks or Azure infrastructure issue you diagnosed and resolved.

Create a job alert for this search

Azure Databricks Platform engineer • Toronto, ON, Canada