Azure Data Architect

Azure Data Architect

Brampton - Hybrid (2-3 days from Office)

Experience Range: 8+ years

1) Architecture & Solution Design

• Define end to end data platform reference architectures using Azure services such as Azure Synapse/SQL, Microsoft Fabric (Lakehouse, Warehouse, Semantic Models), Databricks, Azure Data Lake Storage Gen2, Data Factory/Synapse Pipelines, Event Hubs/Kafka, and Stream Analytics.

• Select appropriate storage and compute patterns (e.g., lakehouse, data mesh, medallion zones, Delta format) aligned to use cases, SLAs, and scalability needs.

• Produce and maintain HLD/LLD, Architecture Decision Records (ADRs), ERDs, and interface/API specifications.

________________________________________

2) Data Modeling & Engineering Patterns

• Establish dimensional (Kimball), 3NF, and Data Vault models where appropriate; promote semantic models for BI (Power BI / Fabric).

• Define ingestion and processing patterns (batch, streaming, CDC, micro batch) and file format standards (Parquet, Delta Lake, Avro).

• Set coding standards and performance guidelines for SQL, T SQL, PySpark, and Spark SQL.

________________________________________

3) Data Integration & Pipelines

• Architect robust ETL/ELT pipelines with ADF/Synapse Pipelines, Fabric Data Factory, Databricks Workflows, and orchestration using ADF triggers and Logic Apps.

• Integrate data from ERPs/CRMs, SaaS applications, APIs, and on prem systems via Self hosted Integration Runtime, Event Hubs, Kafka, Service Bus, and Change Data Capture tools.

________________________________________

4) Governance, Security & Compliance

• Implement data governance (catalogs, lineage, glossary, ownership) using Microsoft Purview; establish data mesh guardrails and federated governance practices.

• Enforce security using Azure AD (Entra ID), RBAC/ABAC, Managed Identities (MSI), Key Vault, Private Endpoints, VNet integration, row/column level security, dynamic data masking, and encryption at rest/in transit.

• Define data quality, master/reference data practices, and support audit, retention, and regulatory requirements.

________________________________________

5) Performance, Reliability & Cost Optimization

• Design for scalability and resilience (partitioning, caching, Z ordering, concurrency control, workload isolation).

• Tune Spark and SQL workloads; optimize storage layouts; configure auto optimize/auto compaction strategies (e.g., Delta).

• Own FinOps for data platforms: capacity planning, cost governance, tagging, budgets/alerts, and rightsizing.

________________________________________

6) DevOps, CI/CD & Observability

• Establish CI/CD for data and infrastructure using Azure DevOps or GitHub Actions, with Terraform/Bicep and deployment strategies (blue green, canary).

• Implement monitoring and alerting with Azure Monitor, Log Analytics, Fabric/Databricks observability, and custom dashboards; define SLOs/SLIs for data pipelines.

• Define backup & disaster recovery strategies and RPO/RTO targets for critical datasets.

________________________________________

7) Leadership & Collaboration

• Lead data architecture governance and design reviews; mentor data engineers and analysts.

• Partner with Product, Security, Enterprise Architecture, and Business stakeholders to align roadmaps and priorities.

• Drive vendor/tool evaluations, proofs of concept, and build vs buy decisions.

________________________________________

Skills & Experience

Required

• 7+ years in data architecture/engineering, including 3–5 years with Azure data services.

• Hands on expertise with several of the following:

o Azure Synapse Analytics / Azure SQL

o Microsoft Fabric (Lakehouse, Warehouse, Semantic Models / DAX)

o Databricks (Spark, Delta Lake, Unity Catalog), PySpark/Spark SQL

o ADLS Gen2, Data Factory / Synapse Pipelines / Fabric Data Factory

o Event Hubs, Service Bus, Kafka, Stream Analytics, CDC tools

• Strong data modeling (dimensional, 3NF, Data Vault) and ELT/ETL design.

• Proficient in SQL, T SQL, PySpark; solid understanding of metadata management and lineage.

• Security & networking basics: AAD, RBAC, VNets, Private Endpoints, Key Vault, Managed Identities.

• IaC & DevOps: Terraform/Bicep, Git, Azure DevOps/GitHub, automated testing for data pipelines.

Preferred

• Experience with data mesh and lakehouse patterns; MLOps handoffs (Feature Store, Model Registry).

• Performance engineering at large scale (TB–PB).

• Knowledge of data quality frameworks (e.g., Great Expectations, Deequ) and DQ scorecards.

• Exposure to Microsoft Purview (catalog, classifications, policies).

• Familiarity with privacy/compliance (SOX, HIPAA, GDPR, PCI).