Snr. Application Developer (Databricks)

Position Description:

This role is hybrid and requires you to be at our Client office at a minimum 1-2 days per week - subject to change at any time.

We are seeking a Senior Databricks Data Engineer to build and evolve enterprise-grade data warehousing and lakehouse capabilities for Capital Markets – Corporate & Investment Banking. You will design robust dimensional data models, implement scalable ingestion and transformation pipelines on Databricks, and enforce strong data governance (Unity Catalog) and data quality (DQE / Expectations) across the platform.
This role requires deep hands-on delivery experience and the ability to work closely with product owners, risk/compliance stakeholders, and downstream consumers (BI, analytics, regulatory reporting).

Your future duties and responsibilities:

Data Warehousing & Data Modeling
• Design and deliver data warehouse / lakehouse models (star schemas, conformed dimensions, facts, and aggregates) for CIB use cases (e.g., revenue, exposure, limits, liquidity, client/profitability, deal pipelines).
• Establish modeling standards for Bronze / Silver / Gold (raw → conformed → curated marts) and ensure consistency across domains.
• Implement incremental loading, SCD patterns (Type 1/2), deduplication, and reconciliation logic suitable for financial services controls.
Databricks Engineering (Lakehouse Implementation)
• Build and maintain scalable ELT/ETL pipelines using Databricks (Spark / PySpark / SQL) and Delta Lake.
• Develop ingestion patterns using Auto Loader (cloudFiles), structured streaming where appropriate, and batch orchestration for daily/monthly financial cycles.
• Optimize Delta tables using best practices (partition strategy, OPTIMIZE, ZORDER, file sizing, caching) and support query performance for downstream BI and analytics workloads.
Governance & Security (Unity Catalog)
• Implement and operationalize governance using Unity Catalog, including:
o Catalog/schema/table design aligned to domains and environments (dev/test/prod)
o Fine-grained permissions (catalog/schema/table/column)
o Row-level and column-level protection (where applicable)
o Auditing and lineage readiness for regulated environments
• Partner with security and compliance to ensure appropriate access models for sensitive CIB datasets.
Data Quality & Controls (DQE / Expectations)
• Define and enforce Data Quality Expectations using Databricks DQE / Delta Live Tables expectations (or equivalent controls framework).
• Implement DQ controls such as:
o Null checks, type checks, referential integrity, range validations
o Duplicate detection and key constraints
o Reconciliation (source-to-target balancing, financial totals validation)
• Publish quality metrics and operational alerts; support SLA reporting and production readiness standards.
Delivery & Stakeholder Engagement
• Help team translate business requirements into data products in collaboration with business stakeholders, analysts, and architects.
• Produce clear technical documentation (data definitions, lineage, runbooks, operational procedures).
• Support production operations, incident triage, root-cause analysis, and continuous improvement.
________________________________________

Required qualifications to be successful in this role:

• Over 5 years of data engineering experience in enterprise environments; financial services experience strongly preferred.
• Strong background in data warehousing and dimensional modeling (facts/dimensions, star schema, SCD, data marts).
• Hands-on expertise with Databricks, including Delta Lake, Spark/PySpark, and Databricks SQL.
• Demonstrated experience implementing governance and access control using Unity Catalog.
• Demonstrated experience implementing data quality frameworks using DQE / DLT expectations or equivalent (Great Expectations, Deequ, custom rules engines).
• Strong SQL skills and experience working with large-scale structured data (RDBMS and/or cloud data platforms).
• Solid software engineering practices: version control (Git), CI/CD concepts, testing, code reviews.
________________________________________
Preferred Qualifications
• Experience with orchestration tools (Databricks Workflows, Airflow, ADF, etc.).
• Experience designing data products for BI tools (Power BI/Tableau), semantic layers, and performant SQL marts.


"CGI is providing a reasonable estimate of the pay range for this role. The determination of this range includes factors such as skill set level, geographic market, experience and training, and licenses and certifications. Compensation decisions depend on the facts and circumstances of each case. A reasonable estimate of the current range is $95,–$,. This role is an existing vacancy."

#LI-AV1

Skills:

• Code Review
• Code Review
• Database Design
• Performance/Stress Testing
• Software Architecture
• Software Design Patterns
• Financial Services