Big data Offres d'emploi - Oliver, BC

The Tech Lead, Data Engineering

Position Overview :

We are seeking a highly skilled, experienced and hands-on Senior Data Engineer to serve as a Tech Lead in a critical data implementation project. The ideal candidate will have 7 to 10 years of experience in ETL, Databricks, Python, Spark, and SQL databases and a strong understanding of data ingestion, transformation, and implementation in complex environments. This role requires expertise in DevOps and CI / CD for data, advanced SQL and Python programming skills, and hands-on experience with modern data engineering tools and architectures.

Responsibilities :

• Serve as a Tech Lead in a data implementation project, guiding architecture decisions and best practices.
• Design, build, and optimize ETL / ELT pipelines for batch and streaming data using Databricks, PySpark, Python, and SQL .
• Implement dimensional modeling and database normalization techniques for efficient data storage and retrieval.
• Lead database development, including DDL and DML scripting, query optimization, and performance profiling across platforms like Spark, PostgreSQL, and Databricks SQL .
• Support the Development and maintaining of CI / CD pipelines for data workflows using Terraform, Kubernetes, Argo, or Cross Plane .
• Work with DBT to manage models, seeds, environments, macros, unit tests, and incremental materialization .
• Implement data versioning and migration strategies using Alembic (preferred), Flyway, or Liquibase .
• Optimize data ingestion and transformation for structured and unstructured data sources.
• Design and implement real-time streaming solutions using Azure EventHub or similar.
• Manage infrastructure as code and containerized environments with Docker and Kubernetes .
• Monitor, troubleshoot, and optimize workflows using logging and querying tools like KQL (Azure) or similar .
• Collaborate with data scientists, analysts, and business teams to ensure data solutions align with business needs.

Skills & Qualifications :

Life at Capgemini

Capgemini supports all aspects of your well-being throughout the changing stages of your life and career. For eligible employees, we offer :

About Capgemini

Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market leading capabilities in AI, cloud and data, combined with its deep industry expertise and partner ecosystem. The Group reported 2023 global revenues of €22.5 billion