Data Engineering Lead and Data EngineersAlltech Consulting Services • Ottawa
Data Engineering Lead and Data Engineers
Alltech Consulting Services • Ottawa
30+ days ago
Job type
- Full-time
Job descriptionBuild and optimize data pipelines for efficient data ingestion, transformation and loading from various sources while ensuring data quality and integrity. Design, develop, and deploy Spark program in databricks environment to process and analyze large volumes of data. Experience of Delta Lake, DWH, Data Integration, Cloud, Design and Data Modelling. Proficient in developing programs in Python and SQL Experience with Data warehouse Dimensional data modeling. Working with event based / streaming technologies to ingest and process data. Working with structured, semi structured and unstructured data. Optimize Databricks jobs for performance and scalability to handle big data workloads. Monitor and troubleshoot Databricks jobs, identify and resolve issues or bottlenecks. Implement best practices for data management, security, and governance within the Databricks environment. Experience designing and developing Enterprise Data Warehouse solutions. Proficient writing SQL queries and programming including stored procedures and reverse engineering existing process. Perform code reviews to ensure fit to requirements, optimal execution patterns and adherence to established standards. 5+ years Python coding experience. 5+ years – SQL Server based development of large datasets 5+ years with Experience with developing and deploying ETL pipelines using Databricks Pyspark. Experience in any cloud data warehouse like Synapse, Big Query, Redshift, Snowflake. Experience in Data warehousing – OLTP, OLAP, Dimensions, Facts, and Data modeling. Previous experience leading an enterprise-wide Cloud Data Platform migration with strong architectural and design skills. Experience with Cloud based data architectures, messaging, and analytics. Cloud certification(s). Strong experience in Code Optimizations and Performance tuning
Job Description
Must Have
- Snowflake
- Databricks
- SQL
- Python
- ADF
- PySpark
- Data Warehouse Concepts
Job Description :
This position is for a Cloud Data engineer with a background in Python, Pyspark, SQL and data warehousing for enterprise level systems. The position calls for someone that is comfortable working with business users along with business analyst expertise in understanding Banking and Capital markets , especially in investment banking would be a big plus. Strong Database fundamentals is a must along with PySpark Coding expertise.
Major Responsibilities :
Skills :
Create a job alert for this search
Data Engineering Lead and Data Engineers • Ottawa
Similar jobs
Data Engineer with AWS, Glue, Lambda, SQL, Python, Redshift.Must have working knowledge in designing and implementing data pipelines on any of the cloud providers (AWS is preferred).Must be able to...Show more
A leading survey platform is seeking a senior data science leader to guide and mentor junior team members while driving machine learning projects.
Candidates should have over 12 years of data scienc...Show more
A leading wellness company in Outaouais is seeking a Senior Developer (Level 3) specializing in data and integrations.This role focuses on backend systems, data pipelines, and database-driven appli...Show more
A leading technology consultancy in Ottawa is seeking an Analytics Architect to leverage AI and data solutions for clients.
The role involves providing technical leadership, engaging in sales proces...Show more
Our client, a leading organization in the public sector, is seeking a.As a SAP SuccessFactors Data Lead, you will be part of the HR and Data Management department supporting SAP HR modules.The idea...Show more
A leading analytics consulting firm in Ontario is seeking a skilled Data Engineer to architect and implement advanced analytics capabilities.
You will work with large data sets and visualization too...Show more
Work in Ottawa is an initiative of Invest Ottawa, the economic development agency with a mandate of facilitating economic growth and job creation for the city of Ottawa.
As a hub for innovation and ...Show more Data Engineer with AWS, Glue, Lambda, SQL, Python, Redshift.Must have working knowledge in designing and implementing data pipelines on any of the cloud providers (AWS is preferred).Must be able to...Show more 
We are hiring a Data Lead to support a large-scale enterprise transformation program delivering a modern Knowledge Management (KM) solution.
In this role, you will lead data-related activities that ...Show more
We’re Hiring : Manager – Systems Engineering.Welcome to TELUS Digital — where innovation drives impact at a global scale.
As an award-winning digital product consultancy and the digital division of T...Show more
We Are • •Accenture Technology – is an exciting place to be! Accenture helps companies rotate to New IT with services that drive continuous quality and efficiency and shape their digital future.We of...Show more
Reviewing, designing, developing ETL jobs to ingest data into Data Lake, load data to data marts;.Parse unstructured data, semi structured data such XML etc.
Design and develop efficient Mapping and...Show more
A public service organization is seeking an experienced Database Architect for a future opportunity.This role focuses on leading assessments and modernizing the enterprise data ecosystem.Candidates...Show more We’re Hiring : Senior Manager – Systems Engineering.Welcome to TELUS Digital — where innovation drives impact at a global scale.
As an award-winning digital product consultancy and the digital divisi...Show more 
The Senior Data Engineer Service Provider will be responsible for leading the delivery of data on data and analytics platform.
These services are at the forefront of building out data engineering pr...Show more
A leading cybersecurity firm is seeking a Director of Engineering to lead its Email Archive portfolio, focusing on compliance and security.
The role involves guiding distributed teams, enhancing pro...Show more
A leading design services firm in Ottawa is seeking a Senior ASIC Design Engineer with over 8 years of experience in ASIC design and complex RTL logic.
In this role, you will translate system-level ...Show more
TekRek has partnered with a fast growing firm specializing in AI enablement and data optimization.Our client works with some of the top tier technology firms in Silcom Valley.With rapid growth and ...Show more
AWS data engineer DWIDC5725169
Compunnel Inc. • ottawa, on, ca
Full-time
Last updated: 2 days ago • Promoted
Senior Staff ML Engineer & Data Leadership (Hybrid)
Tree Trust • Ottawa
Full-time
Last updated: 3 days ago • Promoted
Senior Data Engineer - Integrations & Cloud Platforms
Groupe Nordik • Outaouais
Full-time
Last updated: 18 days ago • Promoted
Lead Data Solution Architect - Analytics & AI
Avanade • Ottawa
Full-time
Last updated: 8 days ago • Promoted
SAP SuccessFactors Data Lead
Experis • Ottawa, ON, Canada
Full-time
Last updated: 8 days ago • Promoted
Data Engineer : Snowflake DW & ELT Visionary
Tiger Analytics • Ottawa
Full-time
Last updated: 18 days ago • Promoted
Data Engineer
Work in Ottawa • Ottawa, Ontario, CA
Full-time
Quick Apply
Last updated: 4 days ago
AWS data engineer DWIDC5725169 - Compunnel Inc.
Compunnel Inc. • ottawa, on, ca
Full-time
Last updated: 2 days ago • Promoted
Data Lead
CGI • Ottawa
Full-time
Last updated: 6 days ago • Promoted
System Engineering Manager - TELUS Digital AI Data Solutions
TELUS Digital AI Data Solutions • ottawa, on, ca
Full-time
Last updated: 19 days ago • Promoted
Data Centre Engineer
Queer Tech • Ottawa
Full-time
Last updated: 6 days ago • Promoted
Python ETL Developer / Data Engineer - Remote
ipvisibility • Ottawa, ON, Canada
Remote
Full-time
Last updated: 30+ days ago • Promoted
Remote Data Architect - Modernize Enterprise Data
Orangutech • Ottawa H2B, ON, Canada
Remote
Part-time
Last updated: 30+ days ago • Promoted
Senior Manager – Systems Engineering - TELUS Digital AI Data Solutions
TELUS Digital AI Data Solutions • ottawa, on, ca
Full-time
Last updated: 20 days ago • Promoted
Senior Data Engineer
freelance.ca • Ottawa, Canada
Full-time
Last updated: 25 days ago • Promoted
Director of Engineering – Global Data & Compliance Platform
Barracuda Networks Inc. • Ottawa H2B, ON, Canada
Remote
Full-time
Last updated: 23 days ago • Promoted
Senior ASIC Design Engineer – AI / HPC SoCs & RTL Expert
BTA Design Services Inc. • Ottawa H2B, ON, Canada
Full-time
Last updated: 23 days ago • Promoted
Senior Analytics Engineer
TekRek • gatineau, qc, ca
Full-time
Last updated: 30+ days ago • Promoted