Data Engineering Lead and Data EngineersAlltech Consulting Services • Ottawa, ON, CA

No longer accepting applications

Data Engineering Lead and Data Engineers

Alltech Consulting Services • Ottawa, ON, CA

17 days ago

Job type

Full-time

Job description

Job Description

Must Have

Snowflake
Databricks
SQL
Python
ADF
PySpark
Data Warehouse Concepts

Job Description

This position is for a Cloud Data engineer with a background in Python, Pyspark, SQL and data warehousing for enterprise level systems. The position calls for someone that is comfortable working with business users along with business analyst expertise in understanding Banking and Capital markets , especially in investment banking would be a big plus. Strong Database fundamentals is a must along with PySpark Coding expertise.

Major Responsibilities

Build and optimize data pipelines for efficient data ingestion, transformation and loading from various sources while ensuring data quality and integrity.

Design, develop, and deploy Spark program in databricks environment to process and analyze large volumes of data.

Experience of Delta Lake, DWH, Data Integration, Cloud, Design and Data Modelling.

Proficient in developing programs in Python and SQL

Experience with Data warehouse Dimensional data modeling.

Working with event based / streaming technologies to ingest and process data.

Working with structured, semi structured and unstructured data.

Optimize Databricks jobs for performance and scalability to handle big data workloads.

Monitor and troubleshoot Databricks jobs, identify and resolve issues or bottlenecks.

Implement best practices for data management, security, and governance within the Databricks environment. Experience designing and developing Enterprise Data Warehouse solutions.

Proficient writing SQL queries and programming including stored procedures and reverse engineering existing process.

Perform code reviews to ensure fit to requirements, optimal execution patterns and adherence to established standards.

Skills

5+ years Python coding experience.

5+ years – SQL Server based development of large datasets

5+ years with Experience with developing and deploying ETL pipelines using Databricks Pyspark.

Experience in any cloud data warehouse like Synapse, Big Query, Redshift, Snowflake.

Experience in Data warehousing – OLTP, OLAP, Dimensions, Facts, and Data modeling.

Previous experience leading an enterprise-wide Cloud Data Platform migration with strong architectural and design skills.

Experience with Cloud based data architectures, messaging, and analytics.

Cloud certification(s).

Strong experience in Code Optimizations and Performance tuning

#J-18808-Ljbffr

Create a job alert for this search

Data Engineering Lead and Data Engineers • Ottawa, ON, CA

Similar jobs

Senior Data Scientist, Power Systems Analytics

Hydroone • Ottawa H2B, ON, Canada

Full-time

A leading electricity provider in Ontario is seeking an experienced Data Scientist to join their team.The role involves leading data science practices, performing analysis on operational data, and ...Show more

Last updated: 3 days ago • Promoted

Head of Hardware Engineering & AI-Driven Innovation

Ross Video • Ottawa H2B, ON, Canada

Full-time

A leading multimedia technology company in Ottawa is seeking a Hardware Development Director to oversee R&D operations and drive engineering excellence. This position requires extensive experience i...Show more

Last updated: 20 days ago • Promoted

ASIC Design Engineer

Myticas Consulting • Ottawa H2B, ON, Canada

Full-time

We are seeking a highly experienced Senior ASIC Design Engineer to join a high-performance hardware development team.This role involves end-to-end ASIC design, from micro-architecture definition th...Show more

Last updated: 7 days ago • Promoted

Senior ASIC Design Engineer for AI / HPC SoCs

BTA Design Services • Ottawa H2B, ON, Canada

Full-time

A leading design services firm in Ottawa is seeking a Senior ASIC Design Engineer to join their team.This role involves translating system-level requirements into architecture, developing synthesis...Show more

Last updated: 7 days ago • Promoted

ASIC Digital Design, Senior Staff Engineer - 13121

Synopsys, Inc. • Ottawa H2B, ON, Canada

Full-time

At Synopsys, we drive the innovations that shape the way we live and connect.Our technology is central to the Era of Pervasive Intelligence, from self-driving cars to learning machines.We lead in c...Show more

Last updated: 9 days ago • Promoted

Hybrid Software Engineering Manager — Lead AI‑Driven SaaS

Magnet Forensics • Ottawa H2B, ON, Canada

Remote

Full-time

A leading technology firm in Ottawa is seeking a Software Engineering Manager to drive the next generation of digital forensics tools. You will lead technical teams, establish modern AI practices in...Show more

Last updated: 1 day ago • Promoted

Senior ASIC Design Engineer - Innovate in Digital Silicon

High Tech Genesis • Ottawa H2B, ON, Canada

Full-time

A design services company in Ottawa is seeking an experienced ASIC Design Engineer.You will be responsible for developing and integrating code while collaborating with systems engineers on architec...Show more

Last updated: 20 days ago • Promoted

Director of Engineering – Global Data & Compliance Platform

Barracuda Networks Inc. • Ottawa H2B, ON, Canada

Remote

Full-time

A leading cybersecurity firm is seeking a Director of Engineering to lead its Email Archive portfolio, focusing on compliance and security. The role involves guiding distributed teams, enhancing pro...Show more

Last updated: 1 day ago • Promoted

Senior Data Engineer

freelance.ca • Ottawa, Canada

Full-time

The Senior Data Engineer Service Provider will be responsible for leading the delivery of data on data and analytics platform. These services are at the forefront of building out data engineering pr...Show more

Last updated: 3 days ago • Promoted

Staff Machine Learning Engineer & Tech Leader

SurveyMonkey • Ottawa H2B, ON, Canada

Remote

Full-time

A leading survey platform is looking for an experienced Data Science Manager to lead a team and drive product improvements. This role involves mentoring junior data scientists, promoting data-driven...Show more

Last updated: 1 day ago • Promoted

Senior ASIC Design Engineer for AI / HPC SoCs

Btadesignservices • Ottawa H2B, ON, Canada

Full-time

A leading semiconductor firm located in Ottawa is searching for a Senior ASIC Design Engineer.The role requires a minimum of 8 years in ASIC design with strong skills in RTL logic and lab troublesh...Show more

Last updated: 6 days ago • Promoted

Senior Go Engineer : Cloud Native, Kubernetes & Tech Lead

Flexera • Ottawa H2B, ON, Canada

Full-time

A leading technology firm in Ottawa is seeking a Software Engineer with a strong background in Go and relevant experience in container technologies like Terraform and Kubernetes.The role involves d...Show more

Last updated: 8 days ago • Promoted

Senior ASIC Design Engineer

Btadesignservices • Ottawa H2B, ON, Canada

Full-time

Headquartered in Ottawa, BTA Design serves the electronic design community from Canada, specializing in ASIC and FPGA design and verification, and embedded software development services.We augment ...Show more

Last updated: 6 days ago • Promoted

Senior ASIC Design Engineer - SystemVerilog & Low Power

High Tech Genesis Inc. • Ottawa H2B, ON, Canada

Full-time

A leading design services company in Ottawa is seeking an experienced ASIC design engineer to collaborate with systems engineers and produce high-quality implementation specifications.The role requ...Show more

Last updated: 1 day ago • Promoted

Head of Supply Chain Data & AI Solutions

Ciena • Ottawa H2B, ON, Canada

Full-time

A leading technology company in Ottawa seeks a Director of Data Services to oversee data strategies within the Supply Chain. You will implement AI-driven solutions, enhance efficiency, and drive dat...Show more

Last updated: 13 days ago • Promoted

Senior ASIC Design Engineer - High-Speed Networking & RTL

Myticas Consulting • Ottawa H2B, ON, Canada

Full-time

A leading consulting firm is seeking a highly experienced Senior ASIC Design Engineer to join their team in Ottawa.The role involves comprehensive ASIC design, from defining micro-architecture to i...Show more

Last updated: 7 days ago • Promoted

Software Engineering Manager - Lead High-Impact Teams

N-able • Ottawa H2B, ON, Canada

Remote

Full-time

A leading cybersecurity firm in Canada is seeking a Software Engineering Manager to oversee team delivery of software solutions. The position involves managing and mentoring engineers, defining proj...Show more

Last updated: 1 day ago • Promoted

Senior ASIC Design Engineer – AI / HPC SoCs & RTL Expert

BTA Design Services Inc. • Ottawa H2B, ON, Canada

Full-time

A leading design services firm in Ottawa is seeking a Senior ASIC Design Engineer with over 8 years of experience in ASIC design and complex RTL logic. In this role, you will translate system-level ...Show more

Last updated: 1 day ago • Promoted