TPM- Data Engineering

Job Description

Technical Program Manager (TPM) – Databricks Platform

Role Overview

We are seeking an experienced Technical Program Manager (TPM) with strong expertise in Databricks and modern data platforms to lead and drive large-scale data & AI transformation programs. This role will work at the intersection of business, data engineering, and platform architecture, ensuring successful delivery of enterprise-grade data solutions on Databricks.

Key Responsibilities

Program Leadership

• Drive end-to-end execution of data platform programs built on Databricks (Lakehouse architecture).
  
• Manage multiple cross-functional teams including data engineering, data science, DevOps, and BI teams.
  
• Own program roadmap, delivery timelines, risk management, and stakeholder communication.
  
• Establish governance frameworks for program delivery, quality, and performance tracking.
  

Databricks Platform Ownership

• Lead design and implementation of Databricks-based data platforms (Azure Databricks / AWS Databricks / GCP).
  
• Ensure best practices in: Delta Lake
  Data pipelines (batch + streaming)
  Medallion architecture (Bronze, Silver, Gold layers)
  
• Drive platform optimization including cost, performance, and scalability.
  

Technical Strategy & Execution

• Collaborate with architects to define data architecture standards and patterns.
  
• Oversee development of: ETL/ELT pipelines
  Data ingestion frameworks
  Data transformation and curation layers
  
• Ensure integration with ecosystem tools like: Azure Data Factory / Airflow
  Power BI / Tableau
  MLflow, Feature Store
  

Stakeholder & Business Alignment

• Act as a bridge between business leaders and engineering teams.
  
• Translate business requirements into technical execution plans.
  
• Provide leadership updates and executive reporting on program progress.
  

Delivery Excellence

• Define and track KPIs, SLAs, and program success metrics.
  
• Identify and mitigate risks, dependencies, and bottlenecks.
  
• Ensure adherence to Agile / Scrum / SAFe methodologies.
  

Data Governance & Security

• Ensure compliance with: Data governance policies
  Security and privacy regulations
  
• Implement best practices for: Data lineage
  Data quality
  Access controls
  

Skills & Qualifications

Mandatory Skills

• 10+ years in technology roles, with 5+ years in program management.
  
• Strong hands-on or working knowledge of: Databricks (must-have)
  Apache Spark / PySpark / SQL
  Delta Lake architecture
  
• Experience in cloud platforms: Azure (preferred), AWS, or GCP
  
• Proven experience delivering enterprise data platform programs.
  

Technical Expertise

• Data Engineering concepts: Data modeling, ETL/ELT, pipelines
  
• Real-time streaming (Kafka / Event Hub)
  
• CI/CD for data pipelines
  
• DataOps / MLOps frameworks
  

Program Management Skills

• Strong experience in: Agile / Scrum / SAFe frameworks
  Program governance and reporting
  
• Excellent stakeholder management skills, including C-level engagement.