Data Engineering Lead and Data EngineersAlltech Consulting Services • Ottawa, ON, CA

Data Engineering Lead and Data Engineers

Alltech Consulting Services • Ottawa, ON, CA

2 days ago

Job type

Full-time

Job description

Job Description

Must Have

Snowflake
Databricks
SQL
Python
ADF
PySpark
Data Warehouse Concepts

Job Description

This position is for a Cloud Data engineer with a background in Python, Pyspark, SQL and data warehousing for enterprise level systems. The position calls for someone that is comfortable working with business users along with business analyst expertise in understanding Banking and Capital markets , especially in investment banking would be a big plus. Strong Database fundamentals is a must along with PySpark Coding expertise.

Major Responsibilities

Build and optimize data pipelines for efficient data ingestion, transformation and loading from various sources while ensuring data quality and integrity.

Design, develop, and deploy Spark program in databricks environment to process and analyze large volumes of data.

Experience of Delta Lake, DWH, Data Integration, Cloud, Design and Data Modelling.

Proficient in developing programs in Python and SQL

Experience with Data warehouse Dimensional data modeling.

Working with event based / streaming technologies to ingest and process data.

Working with structured, semi structured and unstructured data.

Optimize Databricks jobs for performance and scalability to handle big data workloads.

Monitor and troubleshoot Databricks jobs, identify and resolve issues or bottlenecks.

Implement best practices for data management, security, and governance within the Databricks environment. Experience designing and developing Enterprise Data Warehouse solutions.

Proficient writing SQL queries and programming including stored procedures and reverse engineering existing process.

Perform code reviews to ensure fit to requirements, optimal execution patterns and adherence to established standards.

Skills

5+ years Python coding experience.

5+ years – SQL Server based development of large datasets

5+ years with Experience with developing and deploying ETL pipelines using Databricks Pyspark.

Experience in any cloud data warehouse like Synapse, Big Query, Redshift, Snowflake.

Experience in Data warehousing – OLTP, OLAP, Dimensions, Facts, and Data modeling.

Previous experience leading an enterprise-wide Cloud Data Platform migration with strong architectural and design skills.

Experience with Cloud based data architectures, messaging, and analytics.

Cloud certification(s).

Strong experience in Code Optimizations and Performance tuning

#J-18808-Ljbffr

Create a job alert for this search

Lead Data Engineer • Ottawa, ON, CA

Similar jobs

Enterprise Data Architect - Modernization & Strategy Lead

Orangutech • Ottawa H2B, ON, Canada

Remote

Full-time

A public service organization is seeking an experienced Database Architect for a future opportunity in Ottawa.You will lead the assessment and modernization of the enterprise data ecosystem, focusi...Show more

Last updated: 2 hours ago • Promoted • New!

Hybrid Enterprise Architect : Data‑Driven Systems & Cloud

Thales • Ottawa H2B, ON, Canada

Remote

Full-time

A leading technology firm in Ontario is seeking an experienced Enterprise Architect to visualize and design data-centric systems. The role entails significant responsibilities in developing system a...Show more

Last updated: 5 days ago • Promoted

AI & Data Leader : Solution Architect & Tech Lead

EY • Ottawa

Full-time

A leading global consulting firm in Ottawa seeks a Manager - Solution Architect / Tech Lead to enhance client capabilities in data analytics. This role involves supporting key projects, building inter...Show more

Last updated: 2 days ago • Promoted

Data Analytics Engineer - Hybrid

Ribbon Communications Inc. • Ottawa

Full-time

Data Analytics Engineer - Hybrid page is loaded## Data Analytics Engineer - Hybridlocations : Canada, Ontario, Ottawatime type : Full timeposted on : Posted Todayjob requisition id : REQ-2025-2...Show more

Last updated: 30+ days ago • Promoted

Head of Hardware Engineering & AI-Driven Innovation

Ross Video • Ottawa H2B, ON, Canada

Full-time

A leading multimedia technology company in Ottawa is seeking a Hardware Development Director to oversee R&D operations and drive engineering excellence. This position requires extensive experience i...Show more

Last updated: 5 days ago • Promoted

Associate Data Engineer

Export Development Canada • Ottawa

Full-time +1

At EDC, we support Canadian businesses to succeed globally.We provide the financial tools and expertise they need to explore new markets, reduce risks, all towards the goal of making Canada and the...Show more

Last updated: 10 days ago • Promoted

Senior Fullstack Engineer, Data Foundations & Tooling

Fullscript • Ottawa

Full-time

A leading health technology company in Ottawa is seeking a Senior Fullstack Developer to join their Hercules team.In this role, you will design and build backend services, improve data flows, and c...Show more

Last updated: 9 days ago • Promoted

Relocate to Malta Data Engineer (Consulting / Big 4)

Black Pen Recruitment • Ottawa, ON, Canada

Full-time +1

Our client is a Microsoft Gold Partner and leader in Microsoft software implementations for medium to large organisations, providing their clients with the ability to keep up with the ever-changing...Show more

Last updated: 30+ days ago • Promoted

Data Engineering Lead and Data Engineers

Alltech Consulting Services • Ottawa

Full-time

Last updated: 1 day ago • Promoted

Remote Data Architect - Modernize Enterprise Data

Orangutech • Ottawa H2B, ON, Canada

Remote

Full-time

A public service organization is seeking an experienced Database Architect for a future opportunity.This role focuses on leading assessments and modernizing the enterprise data ecosystem.Candidates...Show more

Last updated: 30+ days ago • Promoted

Data Lead / GPO

Accion Labs • Ottawa

Full-time

HR Specialist | Talent Acquisition | End to End Recruitment.Proven experience leading data migration from SAP ECC PA / OM into SuccessFactors Employee Central, including developing the data strategy ...Show more

Last updated: 8 days ago • Promoted

Palantir Data Engineer

Quality IT Resourcing • ottawa, on, ca

Full-time

Palantir Data Engineer / Analytics Specialist.There is a growing global demand for professionals with deep expertise in Palantir platforms, particularly within government, defense, and large enterp...Show more

Last updated: 3 hours ago • Promoted • New!

Azure Data Platform Lead | Databricks & Lakehouse Expert

E-Solutions • Ottawa

Full-time

A leading data solutions company in Ottawa seeks a highly skilled Azure Data Engineer to join its team.The ideal candidate will design and optimize large-scale data pipelines, ensuring reliability ...Show more

Last updated: 2 days ago • Promoted

Data Analytics Engineer - Hybrid

Ribbon Communications • Ottawa

Full-time

JOB TITLE Data Analytics Engineer – Ribbon Analytics (Full Time).Ribbon Communications is a global leader in real-time communications, transforming networks to secure IP and cloud-based architec...Show more

Last updated: 30+ days ago • Promoted

Palantir Data Engineer - ottawa

Quality IT Resourcing • ottawa, on, ca

Full-time

Last updated: 3 hours ago • Promoted • New!

Data Engineer - Modern Data Pipelines (Hybrid)

Export Development Canada • Ottawa

Full-time

Export Development Canada is seeking an Associate Data Engineer to join their team in Ottawa.The successful candidate will design and implement data pipelines and support data-centric applications ...Show more

Last updated: 30+ days ago • Promoted

Data Architect

TEEMA Solutions Group • Ottawa

Full-time

Responsibilities may include, but are not limited to : .Collaborate with internal and external stakeholders (Business, IITB, System Integrators) to understand data requirements and use cases for proj...Show more

Last updated: 30+ days ago • Promoted

Senior Data Engineer, AI Solutions ( 12 months Contract)

Kinaxis • Ottawa

Full-time +1

Senior Data Engineer, AI Solutions (12 months Contract).Be among the first 25 applicants.Elevate your career journey by embracing a new challenge with Kinaxis. We are experts in tech, but it’s reall...Show more

Last updated: 9 days ago • Promoted